Microservices in C# Part 3: Queue Pool Sizing

Fork me on GitHub

Fine tuning QueuePool

Fine tuning QueuePool

This tutorial expands on the previous tutorial, focusing on the Queue Pool concept. By way of quick refresher, a Queue Pool is a feature of the Daishi.AMQP library that allows AMQP Queues to be shared among clients in a concurrent capacity, such that each Queue will have 0…1 consumers only. The concept is not unlike database connection-pooling.

We’ve built a small application that leverages a simple downstream Microservice, implements the AMQP protocol over RabbitMQ, and operates a QueuePool mechanism. We have seen how the QueuePool can retrieve the next available Queue:

var queue = QueuePool.Instance.Get();

And how Queues can be returned to the QueuePool:

QueuePool.Instance.Put(queue);

We have also considered the QueuePool default Constructor, how it leverages the RabbitMQ Management API to return a list of relevant Queues:

        private QueuePool(Func<AMQPQueue> amqpQueueGenerator) {
            _amqpQueueGenerator = amqpQueueGenerator;
            _amqpQueues = new ConcurrentBag<AMQPQueue>();

            var manager = new RabbitMQQueueMetricsManager(false, "localhost", 15672, "paul", "password");
            var queueMetrics = manager.GetAMQPQueueMetrics();

            foreach (var queueMetric in queueMetrics.Values) {
                Guid queueName;
                var isGuid = Guid.TryParse(queueMetric.QueueName, out queueName);

                if (isGuid) {
                    _amqpQueues.Add(new RabbitMQQueue {IsNew = false, Name = queueName.ToString()});
                }
            }
        }

Notice the high-order function in the above constructor. In the QueuePool static Constructor we define this function as follows:

        private static readonly QueuePool _instance = new QueuePool(
            () => new RabbitMQQueue {
                Name = Guid.NewGuid().ToString(),
                IsNew = true
            });

This function will be invoked if the QueuePool is exhausted, and there are no available Queues. It is a simple function that creates a new RabbitMQQueue object. The Daishi.AMQP library will ensure that this Queue is created (if it does not already exist) when referenced.

Exhaustion is Expensive

QueuePool exhaustion is something that we need to avoid. If our application frequently consumes all available Queues then the QueuePool will become ineffective. Let’s look at how we go about avoiding this scenario.

First, we need some targets. We need to know how much traffic our application will absorb in order to adequately size our resources. For argument’s sake, let’s assume that our MathController will be subjected to 100,000 inbound HTTP requests, delivered in batches of 10. In other words, at any given time, MathController will service 10 simultaneous requests, and will continue doing so until 100,000 requests have been served.

Stress Testing Using Apache Bench

Apache Bench is a very simple, lightweight tool designed to test web-based applications, and is bundled as part of the Apache Framework. Click here for simple download instructions. Assuming that our application runs on port 46653, here is the appropriate Apache Bench command to invoke 100 MathController HTTP requests in batches of 10:

-ab -n 100 -c 10 http://localhost:46653/api/math/150

Notice the “n” and “c” paramters; “n” refers to “number”, as in the number of requests, and “c” refers to “concurrency”, or the amount of requests to run in simultanously. Running this command will yield something along the lines of the following:

Benchmarking localhost (be patient).....done

Server Software: Microsoft-IIS/10.0
Server Hostname: localhost
Server Port: 46653

Document Path: /api/math/150
Document Length: 5 bytes

Concurrency Level: 10
Time taken for tests: 7.537 seconds
Complete requests: 100
Failed requests: 0
Total transferred: 39500 bytes
HTML transferred: 500 bytes
Requests per second: 13.27 [#/sec] (mean)
Time per request: 753.675 [ms] (mean)
Time per request: 75.368 [ms] (mean, across all concurrent requests)
Transfer rate: 5.12 [Kbytes/sec] received

Connection Times (ms)
min mean[+/-sd] median max
Connect: 0 0 0.4 0 1
Processing: 41 751 992.5 67 3063
Waiting: 41 751 992.5 67 3063
Total: 42 752 992.4 67 3063

Percentage of the requests served within a certain time (ms)
50% 67
66% 1024
75% 1091
80% 1992
90% 2140
95% 3058
98% 3061
99% 3063
100% 3063 (longest request)

Adjusting QueuePool for Optimal Results

Adjusting QueuePool
Those results don’t look great. Incidentally, if you would like more information as regards how to interpret Apache Bench results, click here. Let’s focus on the final section, “Percentage of the requests served within a certain time (ms)”. Here we see that 75% of all requests took just over 1 second (1091 ms) to complete. 10% took over 2 seconds, and 5% took over 3 seconds to complete. That’s quite a long time for such a simple operation running on a local server. Let’s run the same command again:

Benchmarking localhost (be patient).....done

Server Software: Microsoft-IIS/10.0
Server Hostname: localhost
Server Port: 46653

Document Path: /api/math/100
Document Length: 5 bytes

Concurrency Level: 10
Time taken for tests: 0.562 seconds
Complete requests: 100
Failed requests: 0
Total transferred: 39500 bytes
HTML transferred: 500 bytes
Requests per second: 177.94 [#/sec] (mean)
Time per request: 56.200 [ms] (mean)
Time per request: 5.620 [ms] (mean, across all concurrent requests)
Transfer rate: 68.64 [Kbytes/sec] received

Connection Times (ms)
min mean[+/-sd] median max
Connect: 0 0 0.4 0 1
Processing: 29 54 11.9 49 101
Waiting: 29 53 11.9 49 101
Total: 29 54 11.9 49 101

Percentage of the requests served within a certain time (ms)
50% 49
66% 54
75% 57
80% 60
90% 73
95% 80
98% 94
99% 101
100% 101 (longest request)

OK. Those results look a lot better. Even the longest request took 101 ms, and 80% of all requests completed in <= 60 ms.

But where does this discrepancy come from? Remember, that on start-up there are no QueuePool Queues. The QueuePool is empty and does not have any resources to distribute. Therefore, inbound requests force QueuePool to create a new Queue in order to facilitate the request, and then reclaim that Queue when the request has completed.

Does this mean that when I deploy my application, the first batch of requests are going to run much more slowly than subsequent requests?

No, that’s where sizing comes in. As with all performance testing, the objective is to set a benchmark in terms of the expected volume that an application will absorb, and to determine that maximum impact that it can withstand, in terms of traffic. In order to sufficiently bootstrap QueuePool, so that it contains an adequate number of dispensable Queues, we can simply include ASP.NET controllers that leverage QueuePool in our performance run.

Suppose that we expect to handle 100 concurrent users over extended periods of time. Let’s run an Apache Bench command again, setting the level of concurrency to 100, with a suitably high number of requests in order to sustain that volume over a reasonably long period of time:

ab -n 1000 -c 100 http://localhost:46653/api/math/100


Percentage of the requests served within a certain time (ms)
50% 861
66% 938
75% 9560
80% 20802
90% 32949
95% 34748
98% 39756
99% 41071
100% 42163 (longest request)

Again, very poor, but expected results. More interesting is the number of Queues now active in RabbitMQ:

New QueuePool Queues

New QueuePool Queues

In my own environment, QueuePool created 100 Queues in order to facilitate all inbound requests. Let’s run the test again, and consider the results:

Percentage of the requests served within a certain time (ms)
50% 497
66% 540
75% 575
80% 591
90% 663
95% 689
98% 767
99% 816
100% 894 (longest request)

These results are much more respectable. Again, the discrepancy between performance runs is due to the fact that QueuePool was not adequately initialised during the first run. However, QueuePool was initialised with 100 Queues, a volume sufficient to facilitate the volume of request that the application is expected to serve. This is simple an example as possible.

Real world performance testing entails a lot more than simply executing isolated commands against single endpoints, however the principal remains the same. We have effectively determined the optimal size necessary for QueuePool to operate efficiently, and can now size it accordingly on application start-up, ensuring that all inbound requests are served quickly and without bias.

Those already versed in the area of Microservices might object at this point. There is only a single instance of our Microservice, SimpleMathMicroservice, running. One of the fundamental concepts behind Microservice design is scalability. In my next article, I’ll cover scaling, and we’ll drive those performance response times into the floor.

Connect with me:

RSSGitHubTwitter
LinkedInYouTubeGoogle+

Microservices in C# Part 2: Consistent Message Delivery

Fork me on GitHub

Microservice Architecture

Microservice Architecture

Ensuring that Messages are Consumed by their Intended Recipient

This tutorial builds on the simple Microservice application that we built in the previous tutorial. Everything looks good so far, but what happens when we release this to production, and our application is consumed by multiple customers? Routing problems and message-correlation issue begin to rear their ugly heads. Our current example is simplistic. Consider a deployed application that performs work that is much more complex than our example.

Now we are faced with a problem; how to ensure that any given message is received by its intended recipient only. Consider the following process flow:

potential for mismatched message-routing

potential for mismatched message-routing

It is possible that outbound messages published from the SimpleMath Microservice may not arrive at the ASP.NET application in the same order in which the ASP.NET application initially published the corresponding request to the SimpleMath Microservice.

RabbitMQ has built-in safeguards against this scenario in the form of Correlation IDs. A Correlation ID is essentially a unique value assigned by the ASP.NET application to inbound messages, and retained throughout the entire process flow. Once processed by the SimpleMath Microservice, the Correlation ID is inserted into the associated response message, and published to the response Queue.

Upon receipt of any given message, the ASP.NET inspects the message contents, extracts the Correlation ID and compares it to the original Correlation ID. Consider the following pseudo-code:

            Message message = new Message();
            message.CorrelationID = new CorrelationID();

            RabbitMQAdapter.Instance.Publish(message.ToJson(), "MathInbound");

            string response;
            BasicDeliverEventArgs args;

            var responded = RabbitMQAdapter.Instance.TryGetNextMessage("MathOutbound", out response, out args, 5000);

            if (responded) {
                Message m = Parse(response);
                if (m.CorrelationID == message.CorrelationID) {
                    // This message is the intended response associated with the original request
                }
                else {
                    // This message is not the intended response, and is associated with a different request
                    // todo: Put this message back in the Queue so that its intended recipient may receive it...
                }
            }
            throw new HttpResponseException(HttpStatusCode.BadGateway);

What’s wrong with this solution?

It’s possible that any given message may be bounced around indefinitely, without ever reaching its intended recipient. Such a scenario is unlikely, but possible. Regardless, it is likely, given multiple Microservices, that messages will regularly be consumed by Microservices to whom the message was not intended to be delivered. This is an obvious inefficiency, and very difficult to control from a performance perspective, and impossible to predict in terms of scaling.

But this is the generally accepted solution. What else can we do?

An alternative, but discouraged solution is to invoke a dedicated Queue for each request:

dedicated queue per inbound request

dedicated queue per inbound request

Whoa! Are you suggesting that we create a new Queue for each request?!?

Yes, so let’s park that idea right there – it’s essentially a solution that won’t scale. We would place an unnecessary amount of pressure on RabbitMQ in order to fulfil this design. A new Queue for every inbound HTTP request is simply unmanageable.

Or, is it?

What if we could manage this? Imagine a dedicated pool of Queues, made available to inbound requests, such that each Queue was returned to the pool upon request completion. This might sound far-fetched, but this is essentially the way that database connection-pooling works. Here is the new flow:

consistent message routing using queue-pooling

consistent message routing using queue-pooling

Let’s walk through the code, starting with the QueuePool itself:

    public class QueuePool {
        private static readonly QueuePool _instance = new QueuePool(
            () => new RabbitMQQueue {
                Name = Guid.NewGuid().ToString(),
                IsNew = true
            });

        private readonly Func<AMQPQueue> _amqpQueueGenerator;
        private readonly ConcurrentBag<AMQPQueue> _amqpQueues;

        static QueuePool() {}

        public static QueuePool Instance { get { return _instance; } }

        private QueuePool(Func<AMQPQueue> amqpQueueGenerator) {
            _amqpQueueGenerator = amqpQueueGenerator;
            _amqpQueues = new ConcurrentBag<AMQPQueue>();

            var manager = new RabbitMQQueueMetricsManager(false, "localhost", 15672, "guest", "guest");
            var queueMetrics = manager.GetAMQPQueueMetrics();

            foreach (var queueMetric in queueMetrics.Values) {
                Guid queueName;
                var isGuid = Guid.TryParse(queueMetric.QueueName, out queueName);

                if (isGuid) {
                    _amqpQueues.Add(new RabbitMQQueue {IsNew = false, Name = queueName.ToString()});
                }
            }
        }

        public AMQPQueue Get() {
            AMQPQueue queue;

            var queueIsAvailable = _amqpQueues.TryTake(out queue);
            return queueIsAvailable ? queue : _amqpQueueGenerator();
        }

        public void Put(AMQPQueue queue) {
            _amqpQueues.Add(queue);
        }
    }

QueuePool is a static class that retains a reference to a synchronised collection of Queue objects. The most important aspect of this is that the collection is synchronised, and therefore thread-safe. Under the hood, incoming HTTP requests obtain mutually exclusive locks in order to extract a Queue from the collection. In other words, any given request that extracts a Queue is guaranteed to have exclusive access to that Queue.

Note the private constructor. Upon start-up (QueuePool will be initialised by the first inbound HTTP request) and will invoke a call to the RabbitMQ HTTP API, returning a list of all active Queues. You can mimic this call as follows:

curl -i -u guest:guest http://localhost:15672/api/queues

The list of returned Queue objects is filtered by name, such that only those Queues that are named in GUID-format are returned. QueuePool expects that all underlying Queues implement this convention in order to separate them from other Queues leveraged by the application.

Now we have a list of Queues that our QueuePool can distribute. Let’s take a look at our updated Math Controller:

            var queue = QueuePool.Instance.Get();
            RabbitMQAdapter.Instance.Publish(string.Concat(number, ",", queue.Name), "Math");

            string message;
            BasicDeliverEventArgs args;

            var responded = RabbitMQAdapter.Instance.TryGetNextMessage(queue.Name, out message, out args, 5000);
            QueuePool.Instance.Put(queue);

            if (responded) {
                return message;
            }
            throw new HttpResponseException(HttpStatusCode.BadGateway);

Let’s step through the process flow from the perspective of the ASP.NET application:

  1. Retrieves exclusive use of the next available Queue from the QueuePool
  2. Publishes the numeric input (as before) to SimpleMath Microservice, along with the Queue-name
  3. Subscribes to the Queue retrieved from QueuePool, awaiting inbound messages
  4. Receives the response from SimpleMath Microservice, which published to the Queue specified in step #2
  5. Releases the Queue, which is re-inserted into QueuePool’s underlying collection

Notice the Get method. An attempt is made to retrieve the next available Queue. If all Queues are currently in use, QueuePool will create a new Queue.

Summary

Leveraging QueuePool offers greater reliability in terms of message delivery, as well as consistent throughput speeds, given that we no longer need rely on consuming components to re-queue messages that were intended for other consumers.

It offers a degree of predictable scale – performance testing will reveal the optimal number of Queues that the QueuePool should retain in order to achieve sufficient response times.

It is advisable to determine the optimal number of Queues required by your application, so that QueuePool can avoid creating new Queues in the event of pool-exhaustion, reducing overhead.

Connect with me:

RSSGitHubTwitter
LinkedInYouTubeGoogle+

Microservices in C# Part 1: Building and Testing

Fork me on GitHub

Microservice Architecture

Microservice Architecture

Overview

I’m often asked how to design and build a Microservice framework. It’s a tricky concept, considering loose level of coupling between Microservices. Consider the following scenario outlining a simple business process that consists of 3 sub-processes, each managed by a separate Microservice:

Microservice Architecture core concept

Microservice Architecture core concept

In order to test this process, it would seem that we need a Service Bus, or at the very least, a Service Bus mock in order to link the Microservices. How else will the Microservices communicate? Without a Service Bus, each Microservice is effectively offline, and cannot communicate with any component outside its own context. Let’s examine that concept a little bit further…maybe there is a way that we can establish at least some level of testing without a Service Bus.

A Practical Example

First, let’s expand on the previous tutorial and actually implement a simple Microservice-based application. The application will have 2 primary features:

  • Take an integer-based input and double its value
  • Take a text-based input and reverse it

Both features will be exposed through Microservices. Our first task is to define the Microservice itself. At a fundamental level, Microservices consist of a simple set of functionality:

Anatomy of a Microservice

Anatomy of a Microservice

Consider each Microservice daemon in the above diagram. Essentially, each they consist of

  • a Message Dispatcher (publishes messages to a service bus
  • an Event Listener (receives messages from a service bus
  • Proprietary business logic (to handle inbound and outbound messages)

Let’s design a simple contract that encapsulates this:

    internal interface Microservice {
        void Init();
        void OnMessageReceived(object sender, MessageReceivedEventArgs e);
        void Shutdown();
    }

Each Microservice implementation should expose this functionality. Let’s start with a simple math-based Microservice:

    public class SimpleMathMicroservice : Microservice {
        private RabbitMQAdapter _adapter;
        private RabbitMQConsumerCatchAll _rabbitMQConsumerCatchAll;

        public void Init() {
            _adapter = RabbitMQAdapter.Instance;
            _adapter.Init("localhost", 5672, "guest", "guest", 50);

            _rabbitMQConsumerCatchAll = new RabbitMQConsumerCatchAll("Math", 10);
            _rabbitMQConsumerCatchAll.MessageReceived += OnMessageReceived;

            _adapter.Connect();
            _adapter.ConsumeAsync(_rabbitMQConsumerCatchAll);
        }

        public void OnMessageReceived(object sender, MessageReceivedEventArgs e) {
            var input = Convert.ToInt32(e.Message);
            var result = Functions.Double(input);

            _adapter.Publish(result.ToString(), "MathResponse");
        }

        public void Shutdown() {
            if (_adapter == null) return;

            if (_rabbitMQConsumerCatchAll != null) {
                _adapter.StopConsumingAsync(_rabbitMQConsumerCatchAll);
            }

            _adapter.Disconnect();
        }
    }

Functionality in Detail

Init()

Establishes a connection to RabbitMQ. Remember, as per the previous tutorial, we need only a single connection. This connection is a TCP pipeline designed to funnel all communications to RabbitMQ from the Microservice, and back. Notice the RabbitMQConsumerCatchAll implementation. Here we’ve decided that in the event of an exception occurring, our Microservice will catch each exception and deal with it accordingly. Alternatively, we could have implemented RabbitMQConsumerCatchOne, which would cause the Microservice to disengage from the RabbitMQ Queue that it is listening to (essentially a Circuit Breaker, which I’ll talk about in a future post). In this instance, the Microservice is listening to a Queue called “Math”, to which messages will be published from external sources.

OnMessageReceived()

Our core business logic, in this case, multiplying an integer by 2, is implemented here. Once the calculation is complete, the result is dispatched to a Queue called “MathResponse”.

Shutdown()

Gracefully closes the underlying connection to RabbitMQ.

Unit Testing

There are several moving parts here. How do we test this? Let’s extract the business logic from the Microservice. Surely testing this separately from the application will result in a degree of confidence in the inner workings of our Microservice. It’s a good place to start.
Here is the core functionality in our SimpleMathMicroservice (Functions class in Daishi.Math):

        public static int Double(int input) {
            return input * 2;
        }

Introducing a Unit Test as follows ensures that our logic behaves as designed:

    [TestFixture]
    internal class MathTests {
        [Test]
        public void InputIsDoubled() {
            const int input = 5;
            var output = Functions.Double(input);

            Assert.AreEqual(10, output);
        }
    }

Now our underlying application logic is sufficiently covered from a Unit Testing perspective. Let’s focus on the application once again.

Entrypoint

How will users leverage our application? Do they interface with the Microservice framework through a UI? No, like all web applications, we must provide an API layer, exposing a HTTP channel through which users interact with our application. Let’s create a new ASP.NET application and edit Global.asax as follows:

            # region Microservice Init
            _simpleMathMicroservice = new SimpleMathMicroservice();
            _simpleMathMicroservice.Init();

            #endregion

            #region RabbitMQAdapter Init

            RabbitMQAdapter.Instance.Init("localhost", 5672, "guest", "guest", 100);
            RabbitMQAdapter.Instance.Connect();

            #endregion

We’re going to run an instance of our SimpleMathMicroservice alongside our ASP.NET application. This is fine for the purpose of demonstration, however each Microservice should run in its own context, as a daemon (*.exe in Windows) in a production equivalent. In the above code snippet, we initialise our SimpleMathMicroservice and also establish a separate connection to RabbitMQ to allow the ASP.NET application to publish and receive messages. Essentially, our SimpleMathService instance will run silently, listening for incoming messages on the “Math” Queue. Our adjacent ASP.NET application will publish messages to the “Math” Queue, and attempt to retrieve responses from SimpleMathService by listening to the “MathResponse” Queue. Let’s implement a new ASP.NET Controller class to achieve this:

    public class MathController : ApiController {
        public string Get(int id) {
            RabbitMQAdapter.Instance.Publish(id.ToString(), "Math");

            string message;
            BasicDeliverEventArgs args;
            var responded = RabbitMQAdapter.Instance.TryGetNextMessage("MathResponse", out message, out args, 5000);

            if (responded) {
                return message;
            }
            throw new HttpResponseException(HttpStatusCode.BadGateway);
        }
    }

Navigating to http://localhost:{port}/api/math/100 will initiate the following process flow:

  • ASP.NET application publishes the integer value 100 to the “Math” Queue
  • ASP.NET application immediately polls the “MathResponse” Queue, awaiting a response from SimpleMathMicroservice
  • SimpleMathMicroservice receives the message, and invokes Math.Functions.Double on the integer value 100
  • SimpleMathService publishes the result to “MathResponse”
  • ASP.NET application receives and returns the response to the browser.

Summary

In this example, we have provided a HTTP endpoint to access our SimpleMathMicroservice, and have abstracted SimpleMathMicroservice’ core logic, and applied Unit Testing to achieve sufficient coverage. This is an entry-level requirement in terms of building Microservices. The next step, which I will cover in Part 2, focuses on ensuring reliable message delivery.

Connect with me:

RSSGitHubTwitter
LinkedInYouTubeGoogle+

Microservices with C# and RabbitMQ

Fork me on GitHub

Microservices in Action

Microservices in Action

Microservices with C# and RabbitMQ

Overview

Microservices are groupings of lightweight services, interconnected, although independent of each other, without direct coupling or dependency.

Microservices allow flexibility in terms of infrastructure; application traffic is routed to collections of services that may be distributed across CPU, disk, machine and network as opposed to a single monolithic platform designed to manage all traffic.

Anatomy of a Microservice

In its simplest form, a Microservice consists of an event-listener and a message-dispatcher. The event-listener polls a service-bus – generally a durable message-queue – and handles incoming messages. Messages consist of instructions bound in metadata and encoded in a data-interchange format such as JSON, or Protobuf.

Anatomy of a Microservice

Anatomy of a Microservice

Daishi.AMQP Key Components

Microservices Class Diagram - New Page

Daishi.AMQP Class Architecture

Connecting to RabbitMQ

Everything starts with an abstraction. The Daishi.AMQP library abstracts all AMQP components, and provides RabbitMQ implementations. First, we need to connect to RabbitMQ. Let’s look at the Connect method in the RabbitMQAdapter class:

        public override void Connect() {
            var connectionFactory = new ConnectionFactory {
                HostName = hostName,
                Port = port,
                UserName = userName,
                Password = password,
                RequestedHeartbeat = heartbeat
            };

            if (!string.IsNullOrEmpty(virtualHost)) connectionFactory.VirtualHost = virtualHost;
            _connection = connectionFactory.CreateConnection();
        }

A connection is established on application start-up, and is ideally maintained for the duration of the application’s lifetime.

Consuming Messages

A single running instance (generally an *.exe, or daemon) can connect to RabbitMQ and consume messages in a single-threaded, blocking manner. However, this is not the most scalable solution. Processes that read messages from RabbitMQ must subscribe to a Queue, or Exchange. Once subscribed, RabbitMQ manages message delivery, in terms of even-distribution through round-robin, or biased distribution, depending on your Quality of Service (QOS) configuration. Please refer to this post for a more detailed explanation as to how this works.

For now, consider that our Microservice executable can generate multiple processes, each running on a dedicated thread, to consume messages from RabbitMQ in a parallel manner. The AMQPAdapter class contains a method designed to invoke such processes:

        public void ConsumeAsync(AMQPConsumer consumer) {
            if (!IsConnected) Connect();

            var thread = new Thread(o => consumer.Start(this));
            thread.Start();

            while (!thread.IsAlive)
                Thread.Sleep(1);
        }

Notice the input variable of type “AMQPConsumer”. Let’s take a look at that class in more detail:

        public event EventHandler<MessageReceivedEventArgs> MessageReceived;

        public virtual void Start(AMQPAdapter amqpAdapter) {
            stopConsuming = false;
        }

        public void Stop() {
            stopConsuming = true;
        }

        protected void OnMessageReceived(MessageReceivedEventArgs e) {
            var handler = MessageReceived;
            if (handler != null) handler(this, e);
        }

Essentially, the class contains Start and Stop methods, and an event-handler to handle message-delivery. Like most classes in this project, this is an AMQP abstraction. Here is the RabbitMQ implementation:

        protected void Start(AMQPAdapter amqpAdapter, bool catchAllExceptions) {
            base.Start(amqpAdapter);
            try {
                var connection = (IConnection) amqpAdapter.GetConnection();

                using (var channel = connection.CreateModel()) {
                    if (createQueue) channel.QueueDeclare(queueName, true, false, false, queueArgs);
                    channel.BasicQos(0, prefetchCount, false);

                    var consumer = new QueueingBasicConsumer(channel);
                    channel.BasicConsume(queueName, noAck, consumer);

                    while (!stopConsuming) {
                        try {
                            BasicDeliverEventArgs;
                            var messageIsAvailable = consumer.Queue.Dequeue(timeout, out basicDeliverEventArgs);

                            if (!messageIsAvailable) continue;
                            var payload = basicDeliverEventArgs.Body;

                            var message = Encoding.UTF8.GetString(payload);
                            OnMessageReceived(new MessageReceivedEventArgs {
                                Message = message,
                                EventArgs = basicDeliverEventArgs
                            });

                            if (implicitAck && !noAck) channel.BasicAck(basicDeliverEventArgs.DeliveryTag, false);
                        }
                        catch (Exception exception) {
                            OnMessageReceived(new MessageReceivedEventArgs {
                                Exception = new AMQPConsumerProcessingException(exception)
                            });
                            if (!catchAllExceptions) Stop();
                        }
                    }
                }
            }
            catch (Exception exception) {
                OnMessageReceived(new MessageReceivedEventArgs {
                    Exception = new AMQPConsumerInitialisationException(exception)
                });
            }

Let’s Start from the Start

Connect to a RabbitMQ instance as follows:

            var adapter = RabbitMQAdapter.Instance;

            adapter.Init("hostName", 1234, "userName", "password", 50);
            adapter.Connect();

Notice the static declaration of the RabbitMQAdapter class. RabbitMQ connections in this library are thread-safe; a single connection will facilitate all requests to RabbitMQ.

RabbitMQ implements the concept of Channels, which are essentially subsets of a physical connection. Once a connection is established, Channels, which are logical segments of the underlying Connection, can be invoked in order to interface with RabbitMQ. A single RabbitMQ connection can support up to 65,535 Channels, although I would personally scale out client instances, rather than establish such a high number of Channels. Let’s look at publishing a message to RabbitMQ:

        public override void Publish(string message, string exchangeName, string routingKey,
            IBasicProperties messageProperties = null) {
            if (!IsConnected) Connect();
            using (var channel = _connection.CreateModel()) {
                var payload = Encoding.UTF8.GetBytes(message);

                channel.BasicPublish(exchangeName, routingKey,
                    messageProperties ?? RabbitMQProperties.CreateDefaultProperties(channel), payload);
            }
        }

Notice the _connection.CreateModel() method call. This establishes a Channel to interface with RabbitMQ. The Channel is encapsulated within a using block; once we’ve completed our operation, the Channel may be disposed. Channels are relatively cheap to create, in terms of resources, and may be created and dropped liberally.

Messages are sent in UTF-8, byte-format. Here is how to publish a message to RabbitMQ:

            var message = "Hello, World!";
            adapter.Publish(message, "queueName");

This method also contains overloaded exchangeName and routingKey parameters. These are used to control the flow of messages through RabbitMQ resources. This concept is well documented here.

Now let’s attempt to read our message back from RabbitMQ:

            string output;
            BasicDeliverEventArgs eventArgs;
            adapter.TryGetNextMessage("queueName", out output, out eventArgs, 50);

The tryGetNextMessage method reads the next message from the specified Queue, when available. The method will return false in the event that the Queue is empty, after the specified timeout period has elapsed.

Complete code listing below:

        private static void Main(string[] args) {
            var adapter = RabbitMQAdapter.Instance;

            adapter.Init("hostName", 1234, "userName", "password", 50);
            adapter.Connect();

            var message = "Hello, World!";
            adapter.Publish(message, "queueName");

            string output;
            BasicDeliverEventArgs eventArgs;
            adapter.TryGetNextMessage("queueName", out output, out eventArgs, 50);
        }

Consistent Message Polling

Reading 1 message at a time may not be the most efficient means of consuming messages. I mentioned the AMQPConsumer class at the beginning of this post. The following code outlines a means to continuously read messages from a RabbitMQ Queue:

            var consumer = new RabbitMQConsumerCatchAll("queueName", 10);
            adapter.ConsumeAsync(consumer);

            Console.ReadLine();
            adapter.StopConsumingAsync(consumer);

Note the RabbitMQConsumerCatchAll class instantiation. This class is an implementation of RabbitMQConsumer. All  potential exceptions that occur will be handled by this consumer and persisted back to the client along the same Channel as valid messages. As an alternative, the RabbitMQConsumerCatchOne instance can be leveraged instead. Both classes achieve the same purpose, with the exception of their error-handling logic. The RabbitMQConsumerCatchOne class will disconnect from RabbitMQ should an exception occur.

Summary

The Daishi.AMQP library provides a means of easily interfacing with AMQP-driven queuing mechanisms, with built-in support for RabbitMQ, allowing .NET developers to easily integrate solutions with RabbitMQ. Click here for Part 1 in a tutorial series outlining the means to leverage Daishi.AMQP in your .NET application.

Connect with me:

RSSGitHubTwitter
LinkedInYouTubeGoogle+

Building a Highly Available, Durable in-memory Cache

Overview

Caching strategies have become an integral component in today’s software applications. Distributed computing has resulted in caching strategies that have grown quite complex. Coupled with Cloud computing, caching has become something of a dark art. Let’s walk through the rationale behind a cache, the mechanisms that drive it, and how to achieve a highly available, durable cache, without persisting to disk.

Why We Need a Cache

Providing fast data-access

Data stores are growing larger and more distributed. Caches provide fast read capability and enhanced performance vs. reading from disk. Data distributed across multiple hardware stacks, across multiple geographic locations can be centralised at locations geographically close to application users.

Absorbing traffic surges

Sudden bursts in traffic can cause contention in terms of data-persistence. Storing data in memory removes the overhead involved in disk I/O operations, easing the burden on network resources and application threads.

Augmenting NoSQL

NoSQL has gained traction to the extent that it is now pervasive. Many NoSQL offerings, such as Couchbase, implement an eventual-consistency model; essentially, data will eventually persist to disk at some point after a write operation is invoked. This is an effective big data management strategy, however, it results in potential pitfalls on the consuming application-side. Consider an operation originating from an application that expects data to be written immediately. The application may not have the luxury of waiting until the data eventually persists. Caching the data ensures almost immediate availability.

Another common design in NoSQL technology is to direct both reads and writes, that are associated with the same data segment, to the node on which the data segment resides. This minimises node-hopping and ensures efficient data-flow. Caching can further augment this process by reducing the NoSQL data-store’s requirement to manage traffic by providing a layer of cached metadata before the data-store, minimising resource-consumption. The following design illustrates the basic structure of a managed cache in a hosted environment using Aerospike – a flash optimised, in-memory database:

Distributed Cache

Distributed Cache

 

High Availability and the Cloud

High availability is a principal applied to hosted solutions, ensuring that the system will be online, if even partly, regardless of failure. Failure takes into account not just hardware or software failure, such as disk failure, or out-of-memory exceptions, but also controlled failure, such as machine maintenance.

How Super Data Centers Manage Infrastructure

Data Centers, such as those managed by Amazon Web Services and Microsoft Azure, distribute infrastructure across regions – physical locations separated geographically. Infrastructure contained within each region is further segmented into Availability Zones, or Availability Sets. These are physical groupings of hosted services within hardware stacks – e.g., server racks. Hardware is routinely patched, maintained, and upgraded within Data Centers. This is applied in a controlled manner, such that resources contained within Availability Zone/Set X will not be taken offline at the same time as resources contained within Availability Zone/Set Z.

Durability and the Cloud

To achieve high availability in hosted applications, the applications should be distributed across Availability Zones/Sets, at least. To further enhance the degree of availability, applications can be distributed across separate regions. Consider the following design:

Highly available, durable, cloud-based cache

Highly available, durable, cloud-based cache

When Things Fall Over

Notice that the design provides 8 Cache servers, distributed evenly across both region and availability zone. Thus, should any given Availability Zone fail, 3 Availability Zones will remain online. In the unlikely event that a Data Center fails, and all Availability Zones fail, the second region will remain online – our application can be said to be highly available.

Note that the design includes AWS Simple Queue Service (SQS) to achieve Cross Data Center Data Replication (XDR). The actual implementation, which I will address in an upcoming post, is slightly more complex, and is simplified here for clarity. Enterprise solutions, such as Aerospike and Couchbase offer XDR as a function.

Traffic is load balanced evenly (or in a more suitable manner) across Availability Zones. A Global DNS service, such as AWS Route 53, directs traffic to each region. In situations where all regions and Availability Zones are available, we might consider distributing traffic based on geographic location. Users based in Ireland can be routed to AWS-Dublin, while German users might be routed to AWS-Frankfurt, for example. Route 53 can be configured to distribute all traffic to live regions, should any given region fail entirely.

Taking Things a Step Further by Minimising PCI DSS Exposure

Applications that handle financial data, such as Merchants, must comply with the requirements outlined by the PCI Data Security Standard. These requirements apply based on your application configuration. For example, storing payment card details on disk requires a higher level of adherence to PCI DSS than offloading the storage effort to a 3rd party.

Requirements for Handling Financial Data

The PCI DSS define data as 2 logical entities; data-in-transit and data-at-rest. Data-at-rest is essentially data that has been persisted to a data-store. Data-in-transit applies to data stored in RAM, although the requirements do not specify that this data must be transient – that it must have a point of origin and a destination. Therefore, storing data in RAM would, at least from a legal-perspective, result in a reduced level of PCI DSS exposure, in that requirements pertaining to storing data on disk, such as encryption, do not apply.

Of course, this raises the question; should sensitive data always be persisted to hard-storage? Or, is storing data in a highly available and durable cache sufficient? I suspect at this point that you might feel compelled to post a strongly-worded comment outlining that this idea is ludicrous – but is it really? Can an in-memory cache, once distributed and durable enough to withstand multiple degrees of failure, operate with the same degree of reliability as a hard data-store? I’d certainly like to prove the concept.

Summary

Caching data allows for increased throughput and optimised application performance. Enhancing this concept further, by distributing your cache across physical machine-boundaries, and further still across multiple geographical locations, results in a highly available, durable in-memory storage mechanism.

Hosting cache servers within close proximity to your customers allows for reduced latency and an enhanced user-experience, as well as providing for several degrees of failure; from component, to software, to Availability Zone/Set, to entire region failure.

Connect with me:

RSSGitHubTwitter
LinkedInYouTubeGoogle+

Object Oriented, Test Driven Design in C# and Java: A Practical Example Part #5

Download the code in C#
Download the code in Java

Check out my interview on .NET Rocks! – TDD on .NET and Java with Paul Mooney

For a brief overview, please refer to this post.

Overview

In the last tutorial we focused on correcting some logic in our classes and tests. It’s about time that we started building Robots. Let’s start with a simple example consisting of the following components:

  • Head
  • Torso
  • 2x Arms
  • 2x Legs

Not the most exciting contraption, but we can expand on this later. For now, let’s define these simple properties and combine them in a simple class called Robot:

C#

    public abstract class Robot {
        public Head Head { get; set; }
        public Torso Torso { get; set; }
        public Arm LeftArm { get; set; }
        public Arm RightArm { get; set; }
        public Leg LeftLeg { get; set; }
        public Leg RightLeg { get; set; }
    }

Java

public abstract class robot {
    private head _head;
    private torso _torso;
    private arm _leftArm;
    private arm _rightArm;
    private leg _leftLeg;
    private leg _rightLeg;

    public head getHead() {
        return _head;
    }

    public void setHead(head head) {
        _head = head;
    }

    public torso getTorso() {
        return _torso;
    }

    public void setTorso(torso torso) {
        _torso = torso;
    }

    public arm getLeftArm() {
        return _leftArm;
    }

    public void setLeftArm(arm leftArm) {
        _leftArm = leftArm;
    }

    public arm getRightArm() {
        return _rightArm;
    }

    public void setRightArm(arm rightArm) {
        _rightArm = rightArm;
    }

    public leg getleftLeg() {
        return _leftLeg;
    }

    public void setLeftLeg(leg leftLeg) {
        _leftLeg = leftLeg;
    }

    public leg getRightLeg() {
        return _rightLeg;
    }

    public void setRightLeg(leg rightLeg) {
        _rightLeg = rightLeg;
    }
}

“Wait! Why are you writing implementation-specific code? This is about TDD! Where are your unit tests?”

If I write a class as above, I can expect that it will work because it’s essentially a template, or placeholder for data. There is no logic, and very little, if any scope for error. I could write unit tests for this class, but what would they prove? There is nothing specific to my application, in terms of logic. Any associated unit tests would simply test the JVM (Java) or CLR (.NET), and would therefore be superfluous.

Disclaimer: A key factor in mastering either OOD or TDD is knowing when not to use them.

Let’s start building Robots. Robots are complicated structures composed of several key components. Our application might grow to support multiple variants of Robot. Imagine an application that featured thousands of Robots. Assembling each Robot to a unique specification would be a cumbersome task. Ultimately, the application would become bloated with Robot bootstrapper code, and would quickly become unmanageable.

“Sounds like a maintenance nightmare. What can we do about it?”

Ideally we would have a component that created each Robot for us, with minimal effort. Fortunately, from a design perspective, a suitable pattern exists.

Introducing the Builder Pattern

We're here to build your robots, sir!

We’re here to build your robots, sir!

The Builder pattern provides a means to encapsulate the means by which an object is constructed. It also allows us to modify the construction process to allow for multiple implementations; in our case, to create multiple variants of Robot. In plain English, this means that an application the leverages a builder component does not need to know anything about the object being constructed.

Builder Design Pattern

Builder Design Pattern

“That sounds great, but isn’t the Builder pattern really just about good house-keeping? All we really achieve here is separation-of-concerns, which is fine, but my application is simple. I just need a few robots; I can assemble these with a few lines of code.”

The Builder pattern is about providing an object-building schematic. Let’s go through the code:

C#

    public abstract class RobotBuilder {
        protected Robot robot;

        public Robot Robot { get { return robot; } }

        public abstract void BuildHead();
        public abstract void BuildTorso();
        public abstract void BuildArms();
        public abstract void BuildLegs();
    }

Java

public abstract class robotBuilder {
    protected robot robot;

    public robot getRobot() {
        return robot;
    }

    public abstract void buildHead();

    public abstract void buildTorso();

    public abstract void buildArms();

    public abstract void buildLegs();
}

This abstraction is the core of our Builder implementation. Notice that it provides a list of methods necessary to construct a Robot. Here is a simple implementation that builds a basic Robot:

C#

    public class BasicRobotBuilder : RobotBuilder {
        public BasicRobotBuilder() {
            robot = new BasicRobot();
        }

        public override void BuildHead() {
            robot.Head = new BasicHead();
        }

        public override void BuildTorso() {
            robot.Torso = new BasicTorso();
        }

        public override void BuildArms() {
            robot.LeftArm = new BasicLeftArm();
            robot.RightArm = new BasicRightArm();
        }

        public override void BuildLegs() {
            robot.LeftLeg = new BasicLeftLeg();
            robot.RightLeg = new BasicRightLeg();
        }
    }

Java

public class basicRobotBuilder extends robotBuilder {

    public basicRobotBuilder() {
        robot = new basicRobot();
    }

    @Override
    public void buildHead() {
        robot.setHead(new basicHead());
    }

    @Override
    public void buildTorso() {
        robot.setTorso(new basicTorso());
    }

    @Override
    public void buildArms() {
        robot.setLeftArm(new basicLeftArm());
        robot.setRightArm(new basicRightArm());
    }

    @Override
    public void buildLegs() {
        robot.setLeftLeg(new basicLeftLeg());
        robot.setRightLeg(new basicRightLeg());
    }
}

It’s not your application’s job to build robots. It’s your application’s job to manage those robots at runtime. The application should be agnostic in terms of how robots are provided. Let’s add another Robot to our application; this time, let’s design the Robot to run on caterpillars, rather than legs.

Caterpillar Robot

Caterpillar Robot

First, we introduce a new class called Caterpillar. Caterpillar must extend Leg, so that it’s compatible with our Robot and RobotBuilder abstractions.

C#

    public class Caterpillar : Leg {}

Java

public class caterpillar extends leg {

}

This class doesn’t do anything right now. We’ll implement behaviour in the next tutorial. For now, let’s provide a means to build our CaterpillarRobot.

C#

    public class CaterpillarRobotBuilder : RobotBuilder {
        public CaterpillarRobotBuilder() {
            robot = new CaterpillarRobot();
        }

        public override void BuildHead() {
            robot.Head = new BasicHead();
        }

        public override void BuildTorso() {
            robot.Torso = new BasicTorso();
        }

        public override void BuildArms() {
            robot.LeftArm = new BasicLeftArm();
            robot.RightArm = new BasicRightArm();
        }

        public override void BuildLegs() {
            robot.LeftLeg = new Caterpillar();
            robot.RightLeg = new Caterpillar();
        }
    }

Java

public class caterpillarRobotBuilder extends robotBuilder {
    public caterpillarRobotBuilder() {
        robot = new caterpillarRobot();
    }

    @Override
    public void buildHead() {
        robot.setHead(new basicHead());
    }

    @Override
    public void buildTorso() {
        robot.setTorso(new basicTorso());
    }

    @Override
    public void buildArms() {
        robot.setLeftArm(new basicLeftArm());
        robot.setRightArm(new basicRightArm());
    }

    @Override
    public void buildLegs() {
        robot.setLeftLeg(new caterpillar());
        robot.setRightLeg(new caterpillar());
    }
}

Notice that all methods remain the same, with the exception of BuildLegs, which now attaches Caterpillar objects to both left and right legs. We create an instance of our CaterpillarRobot as follows:

C#

    var caterpillarRobotBuilder = new CaterpillarRobotBuilder();

    caterpillarRobotBuilder.BuildHead();
    caterpillarRobotBuilder.BuildTorso();
    caterpillarRobotBuilder.BuildArms();
    caterpillarRobotBuilder.BuildLegs();

Java

        caterpillarRobotBuilder caterpillarRobotBuilder = new caterpillarRobotBuilder();
        caterpillarRobotBuilder.buildHead();
        caterpillarRobotBuilder.buildTorso();
        caterpillarRobotBuilder.buildArms();
        caterpillarRobotBuilder.buildLegs();

“That’s still a lot of repetitive code. Your CaterpillarRobot isn’t that much different from your BasicRobot. Why not just extend CaterpillarRobotBuilder from BasicRobotBuilder?”

Yes, both classes are similar. Here, you must use your best Object Oriented judgement. If your classes are unlikely to change, then yes, extending BasicRobotBuilder to CaterpillarRobotBuilder might be a worthwhile strategy. However, you must consider the cost of doing this, should future requirements change. Suppose that we introduce a fundamental change to our CaterpillarRobot class, such that it no longer resembles, nor behaves in the same manner as a BasicRobot. In that case, we would have to extract the CaterpillarRobotBuilder class from BasicRobotBuilder, and extend if from RobotBuilder, which may involve significant effort.
As regards repetitive code, let’s look at a means of encapsulating this further, in what’s called a Director. The Director’s purpose is to invoke the Builder’s methods to facilitate object construction, encapsulating construction logic, and removing the need to implement build methods explicitly:

C#

    public class RobotConstructor {
        public void Construct(RobotBuilder robotBuilder) {
            robotBuilder.BuildHead();
            robotBuilder.BuildTorso();
            robotBuilder.BuildArms();
            robotBuilder.BuildLegs();
        }
    }

Java

    public void Construct(robotBuilder robotBuilder) {
        robotBuilder.buildHead();
        robotBuilder.buildTorso();
        robotBuilder.buildArms();
        robotBuilder.buildLegs();
    }

Now our build logic is encapsulated within a controlling class, which is agnostic in terms of the actual implementation of robotbuilder – we can load any implementation we like, and our constructor will just build it.

C#

            var robotConstructor = new RobotConstructor();
            var basicRobotBuilder = new BasicRobotBuilder();

            robotConstructor.Construct(basicRobotBuilder);

Java

        robotConstructor robotConstructor = new robotConstructor();
        basicRobotBuilder basicRobotBuilder = new basicRobotBuilder();

        robotConstructor.Construct(basicRobotBuilder);

Summary

We’ve looked at the Builder pattern in this tutorial, and have found that it is an effective way of:

  • Providing an abstraction that allows multiple robot types to be assembled in multiple configurations
  • Encapsulates robot assembly logic
  • Facilitates the instantiation of complex, composite objects

In the next tutorial in this series we’ll focus on making robots fight.

Connect with me:

RSSGitHubTwitter
LinkedInYouTubeGoogle+

Protecting ASP.NET Applications Against CSRF Attacks

Fork me on GitHub

ARMOR

ARMOR

For a brief overview of the Encrypted Token Pattern, please refer to this post.

Overview

The Encrypted Token Pattern is a defence mechanism against Cross Site Request Forgery (CSRF) attacks, and is an alternative to its sister-patterns; Synchroniser Token, and Double Submit Cookie.

Each of these patterns have the same objective:

  1. To ensure that any given HTTP request originated from a trustworthy source
  2. To uniquely identify the user that issued the HTTP request

In the first instance, the need to ensure that requests originate from a trustworthy source is an obvious requirement. Essentially, we need to guarantee that any given request has originated not only from the user’s web-browser, but also from a non-malicious link, or connection.

Why the Encrypted Token Pattern?

A Simple CSRF Attack

Consider a banking application. Suppose that the application exposes an API that allows the transfer of funds between accounts as follows:

http://mybank.com/myUserId/fromAccount/toAccount/amount

Web browsers share state, in terms of cookies, across tabs. Imagine a user that is logged into mybank.com. They open a new tab in their internet browser and navigate to a website that contains a link to the above URI. An attacker that knows the user’s bank account number could potentially transfer any given sum from the user’s account to their own. Remember that the user is already logged in to mybank.com at this point, and have an established session on the web-server, if not a persistent cookie in their web-browser. The browser simply opens a new tab, leverages the user’s logged in credentials, and executes the HTTP request on the user’s behalf.

How to Defend Against CSRF Attacks

In order to defend against such attacks, we need to introduce a token on the user’s behalf, and validate that token on the web server during HTTP requests, ensuring that each request

  • Originates from a trusted source
  • Uniquely identifies the user

Why uniquely identify the user? Consider that CSRF attacks can potentially originate from valid users.

A More Sophisticated CSRF Attack

John and David are both valid users of mybank.com. John decides to post a malicious link. This time, the attacks is more sophisticated. John has built a small web server that issues a HTTP request to the mybank.com money-transfer API:

http://mybank.com/myUserId/fromAccount/toAccount/amount

This time, John has supplied a valid token – his own (remember, John is also a valid user). Now, assuming that mybank.com does not validate the identity of the user in the supplied token, it will determine the request to have originated from a trusted source, and allow the transfer to take place.

The Encrypted Token Pattern

The Encrypted Token Patterns protects web applications against CSRF attacks by generating a secure token at server level, and issuing the token to the client. The token itself is essentially a JSON Web Token (JWT) composed of a unique User ID, randomly generated number (nonce), and timestamp. Given that the token is a JSON object, it is possible to include any additional metadata in the token. The process flow is as follows:

Encrypted Token Pattern

Encrypted Token Pattern (click to enlarge)

Leveraging the Encrypted Token Pattern

The Advanced Resilient Mode of Recognition (ARMOR) is a C# implementation of the Encrypted Token Pattern, available on GitHub under the MIT license that provides a means of protecting ASP.NET applications from CSRF attacks, by leveraging the Encrypted Token Pattern. The following steps describes a typical setup configuration.

ARMOR

ARMOR is a framework composed of interconnecting components exposed through custom DelegatingHandler and AuthorizationAttribute classes. ARMOR is essentially an advanced encryption and hashing mechanism, leveraging the Rijndael encryption standard, and SHA256 hashing by default, though these are concrete implementations; ARMOR provides abstractions in terms of encryption, allowing developers to leverage custom concrete implementations. ARMOR has two primary directives:

  • To generate secure ARMOR tokens
  • To validate secure ARMOR tokens

ARMOR Web Framework

The ARMOR Web Framework is a set of components that leverage ARMOR itself, allowing developers to leverage the ARMOR framework in a plug-and-play fashion, without necessarily grappling with the underlying complexities of encryption and hashing. This tutorial focuses on leveraging the ARMOR Web Framework in C# to protect your ASP.NET applications from CSRF attacks.

Leveraging ARMOR in ASP.NET

ARMOR Web Framework Package

Download the ARMOR Web Framework package from Nuget:

PM> Install-Package Daishi.Armor.WebFramework

Apply Configuration Settings

Add the following configuration settings to your web.config file:

<add key=“IsArmed” value=“true” />
<add key=“ArmorEncryptionKey” value=“{Encryption Key}” />
<add key=“ArmorHashKey” value=“{Hashing Key}” />
<add key=“ArmorTimeout” value=“1200000” />

IsArmed

A toggle feature easily allowing developers to turn ARMOR on or off

ArmorEncryptionKey

The encryption key that ARMOR will use to both encrypt and decrypt ARMOR tokens

ArmorHashKey

The hashing key that ARMOR will use to generate and validate hashes contained within ARMOR tokens. ARMOR implements hashes as a means of determining whether or not tokens have been tampered with, and to add an extended level of entropy to token metadata, rendering them more difficult to hijack.

ArmorTimeout

The time in milliseconds that ARMOR Tokens remain valid.

In order to facilitate encryption and hashing, ARMOR requires two keys. You can generate both keys as follows:

byte[] encryptionKey = new byte[32];
byte[] hashingKey = new byte[32];

using (var provider = new RNGCryptoServiceProvider()) {
    provider.GetBytes(encryptionKey);
    provider.GetBytes(hashingKey);
}

These keys must be stored in the ArmorEncryptionKey and ArmorHashKey values in your configuration file, in Base64-format.

Hook the ARMOR Filter to your application

Core Components

Authorization Filter

The Authorization filter reads the ARMOR Token from the HttpRequest Header and validates it against the currently logged in user. Users can be authenticated in any fashion; ARMOR assumes that your user’s Claims are loaded into the current Thread at the point of validation.

The following classes facilitate authorization for both MVC and Web API projects respectively:

  • MvcArmorAuthorizeAttribute
  • WebApiArmorAuthorizeAttribute

 Fortification Filter

The Fortification filter refreshes and re-issues new ARMOR tokens. The following classes facilitate fortification for both MVC and Web API projects respectively:

  • MvcArmorFortifyFilter
  • WebApiArmorFortifyFilter

Generally speaking, it’s ideal that you refresh the incoming ARMOR token for every HTTP request, whether that request validates the Token or not; particularly for GET HTTP requests. Otherwise, the Token may expire unless the user issues a POST, PUT, or DELETE request within the Token’s lifetime.

To do this, simple register the appropriate ARMOR Fortification mechanism in your MVC application,

public static void RegisterGlobalFilters(GlobalFilterCollection filters) {
    filters.Add(new MvcArmorFortifyFilter());
}

or in your Web API application:

config.Filters.Add(new WebApiArmorFortifyFilter());

Now, each HttpResponse issued by your application will contain a custom ARMOR Header containing a new ARMOR Token for use with subsequent HTTP requests:

ArmorResponse

Decorating POST, PUT, and DELETE Endpoints with ARMOR

In an MVC Controller simply decorate your endpoints as follows:

[MvcArmorAuthorize]

And in Web API Controllers:

[WebApiArmorAuthorize]

Integrating Your Application’s Authentication Mechanism

AMROR operates on the basis of Claims and provides default implementations of Claim-parsing components derived from the IdentityReader class in the following classes:

  • MvcIdentityReader
  • WebApiIdentityReader

Both classes return an enumerated list of Claim objects consisting of a UserId Claim. In the case of MVC, the Claim is derived from the ASP.NET intrinsic Identity.Name property, assuming that the user is already authenticated. In the case of Web API, it is assumed that you leverage an instance of ClaimsIdentity as your default IPrincipal object, and that user metadata is stored in Claims held within that ClaimsIdentity. As Such, the WebApiIdentityReader simply extracts the UserId Claim. Both UserId and Timestamp Claims are the only default Claims in an ArmorToken and are loaded upon creation.

If your application leverages a different authentication mechanism, you can simply derive from the default IdentityReader class with your own implementation and extract your logged in user’s metadata, injecting it into Claims necessary for ARMOR to manage. Here is the default Web API implementation.

public override bool TryRead(out IEnumerable<Claim> identity) {
    var claims = new List<Claim>();
    identity = claims;

    var claimsIdentity = principal.Identity as ClaimsIdentity;
    if (claimsIdentity == null) return false;

    var subClaim = claimsIdentity.Claims.SingleOrDefault(c => c.Type.Equals(“UserId”));
    if (subClaim == null) return false;

    claims.Add(subClaim);
    return true;
}

ARMOR downcasts the intrinsic HTTP IPrincipal.Identity object as an instance of ClaimsIdentity and extracts the UserId Claim. Deriving from the IdentityReader base class allows you to implement your own mechanism to build Claims. It’s worth noting that you can store many Claims as you like in an ARMOR Token. ARMOR will decrypt and deserialise your Claims so that they can be read on the return journey back to server from UI.

Adding ARMOR UI Components

The ARMOR WebFramework contains a JavaScript file as follows:

var ajaxManager = ajaxManager || {
    setHeader: function(armorToken) {
    $.ajaxSetup({
        beforeSend: function(xhr, settings) {
            if (settings.type !== “GET”) {
                xhr.setRequestHeader(“Authorization”, “ARMOR “ + armorToken);
            }
        }
    });
}
};

The purpose of this code is to detect the HttpRequest type, and apply an ARMOR Authorization Header for POST, PUT and DELETE requests. You can leverage this on each page of your application (or in the default Layout page) as follows:

<script>
$(document).ready(function () {
ajaxManager.setHeader($(“#armorToken”).val());
});

$(document).ajaxSuccess(function (event, xhr, settings) {
    var armorToken = xhr.getResponseHeader(“ARMOR”) || $(“#armorToken”).val();
    ajaxManager.setHeader(armorToken);
});
</script>

As you can see, the UI contains a hidden field called “armorToken”. This field needs to be populated with an ArmorToken when the page is initially served. The following code in the ARMOR API itself facilitates this:

        public bool TryFortify() {
            var identityReader = identityReaderFactory.Create();
            IEnumerable<Claim> identity;

            var isAuthenticated = identityReader.TryRead(out identity);
            if (!isAuthenticated) return false;

            var claims = identity.ToList();

            var userId = claims.Single(c => c.Type.Equals("UserId")).Value;
            var platform = claims.SingleOrDefault(c => c.Type.Equals("Platform"));

            var encryptionKey = ArmorSettings.EncryptionKey;
            var hashingKey = ArmorSettings.HashingKey;

            var nonceGenerator = new NonceGenerator();
            nonceGenerator.Execute();

            var armorToken = new ArmorToken(userId,
                platform == null ? "ARMOR" : platform.Value,
                nonceGenerator.Nonce);

            var armorTokenConstructor = new ArmorTokenConstructor();
            var standardSecureArmorTokenBuilder =
                new StandardSecureArmorTokenBuilder(armorToken, encryptionKey,
                    hashingKey);
            var generateSecureArmorToken =
                new GenerateSecureArmorToken(armorTokenConstructor,
                    standardSecureArmorTokenBuilder);

            generateSecureArmorToken.Execute();

            httpContext.Response.AppendHeader("ARMOR",
                generateSecureArmorToken.SecureArmorToken);
            return true;
        }

Here we generate the initial ARMOR Token to be served when the application loads. This Token will be leveraged by the first AJAX request and refreshed on each subsequent request. The Token is then loaded into the ViewBag object and absorbed by the associated View:


<div><input id=“armorToken” type=“hidden” value=@ViewBag.ArmorToken /></div>

Now your AJAX requests are decorated with ARMOR Authorization attributes:
ArmorRequest

Summary

Now that you’ve implemented the ARMOR WebFramework, each POST, PUT and DELETE request will persist a Rijndael-encrypted and SHA256-hashed ARMOR Token, which is validated by the server before each POST, PUT, or DELETE request decorated with the appropriate attribute is handled, and refreshed after each request completes. The simple UI components attach new ARMOR Tokens to outgoing requests and read ARMOR Tokens on incoming responses. ARMOR is designed to work seamlessly with your current authentication mechanism to protect your application from CSRF attacks.

Object Oriented, Test Driven Design in C# and Java: A Practical Example Part #4

Download the code in C#
Download the code in Java

Check out my interview on .NET Rocks! – TDD on .NET and Java with Paul Mooney

For a brief overview, please refer to this post.

That’s the hard work done! Our WorkerDrones can now deliver RobotParts to their respective FactoryRoom implementations. Now, we can start building Robots! But wait…

Oh, no. Something's gone wrong!

“Oh, no. Something’s gone horribly wrong!”

It looks like our code is broken. Remember our WorkerDrone logic?

C#

        public void IdentifyRobotPart(RobotPart robotPart) {
            switch (robotPart.RobotPartCategory) {
                case RobotPartCategory.Assembly:
                    _transportMechanism = new AssemblyRoomTransportMechanism();
                    break;
                case RobotPartCategory.Weapon:
                    _transportMechanism = new ArmouryTransportMechanism();
                    break;
            }
        }

Java

    public void identifyRobotPart(robotPart robotPart) {
        switch (robotPart.getRobotPartCategory()) {
            case assembly:
                _transportMechanism = new assemblyRoomTransportMechanism();
                break;
            case weapon:
                _transportMechanism = new armouryTransportMechanism();
                break;
        }
    }

Our Unit Tests only ever considered WorkerDrones that transport a single RobotPart at a time. We haven’t considered the consequences of transporting multiple RobotParts.

As it turns out, our WorkerDrone's TransportMechanism will be overwritten every time a RobotPart is picked up. This will produce incorrect behaviour; essentially, all RobotParts held by a WorkerDrone will be delivered to the same FactoryRoom implementation – the FactoryRoom implementation associated with the last RobotPart to be picked up, which will overwrite the previous RobotPart that was picked up.

“How do we fix this?”

Well, currently our WorkerDrones can only successfully transport a single RobotPart at a time. Our Factory would operate a lot more efficiently if our WorkerDrones could carry multiple RobotParts and successfully deliver them without having to return to the DeliveryBay after each run.

More than 1 RobotPart at a time? Please! Easy work for me!!!

More than 1 RobotPart at a time? Please! Easy work for me!!!

First, let’s change the IdentifyRobotPart method so that it simply returns a TransportMechanism implementation based on the RobotPart parameter:

C#

        public TransportMechanism IdentifyRobotPart(RobotPart robotPart) {
            switch (robotPart.RobotPartCategory) {
                case RobotPartCategory.Assembly:
                    return new AssemblyRoomTransportMechanism((Assembly) robotPart);
                case RobotPartCategory.Weapon:
                    return new ArmouryTransportMechanism((Weapon) robotPart);
            }
            throw new NotImplementedException("I can't identify this component!");
        }

Java

    public transportMechanism identifyRobotPart(robotPart robotPart) throws UnsupportedOperationException {
        switch (robotPart.getRobotPartCategory()) {
            case assembly:
                return new assemblyRoomTransportMechanism((assembly) robotPart);
            case weapon:
                return new armouryTransportMechanism((weapon) robotPart);
        }
        throw new UnsupportedOperationException("I can't identify this component!");
    }

We’ll call this method every time our WorkerDrone picks up a RobotPart. Next, let’s replace our TransportMechanism member-variable with a collection:

C#

    private readonly List<TransportMechanism> _transportMechanisms;

Java

    private List<transportMechanism> _transportMechanisms;

Now we need to modify the OffLoadRobotPart to simply transfer the RobotPart that we just picked up to its associated FactoryRoom instance.

C#

        public FactoryRoom OffLoadRobotPart() {
            if (_factoryRoom == null) {
                EnterRoom();
            }
            _factoryRoom.AddRobotPart(_robotPart);
            return _factoryRoom;
        }

Java

    public E offLoadRobotPart() {
        if (_factoryRoom == null) {
            enterRoom();
        }

        _factoryRoom.addRobotPart(_robotPart);
        return _factoryRoom;
    }

Note that we’ve also changed our TransportMechanism abstraction to accept a RobotPart by default. This is where we’ll load the RobotPart that is picked up by the WorkerDrone.

C#

        protected TransportMechanism(RobotPart robotPart) {
            _robotPart = robotPart;
        }

Java

    protected transportMechanism(U robotPart) {
        _robotPart = robotPart;
    }

Now, the most important part. We need to change the TransportRobotParts method to loop through each TransportMechanism (we’ll have 1 for each RobotPart that we picked up) and execute the OffLoadRobotPart method:

C#

       public void TransportRobotParts() {
            foreach (var transportMechanism in _transportMechanisms) {
                transportMechanism.OffLoadRobotPart();
            }

            _transportMechanisms.Clear();
        }

Java

   public void transportRobotParts() {
        for (Iterator<transportMechanism> i = _transportMechanisms.iterator(); i.hasNext(); ) {
            transportMechanism transportMechanism = i.next();
            transportMechanism.offLoadRobotPart();
        }

        _transportMechanisms.clear();
    }

OK. Now our WorkerDrone can successfully deliver multiple RobotParts to multiple FactoryRooms. Now we need to ensure that it will return to the DeliveryBay once it has delivered its payload.

I'm ready to transport more RobotParts! Back to the DeliveryBay I go...

“I’m ready to transport more RobotParts! Back to the DeliveryBay I go…”

AS usual with TDD, let’s start with the tests, and introduce a new test to assert that multiple RobotParts can be delivered successfully. Our test will cover a scenario where a WorkerDrone delivers multiple RobotParts and then returns to the DeliveryBay.

C#

        public void WorkerDroneReturnsToDeliveryBayAfterDeliveringRobotParts() {
            WorkerDrone workerDrone = new MockedWorkerDrone();

            var randomAssembly = new MockedAssembly();
            var randomWeapon = new MockedWeapon();

            workerDrone.PickUpRobotPart(randomAssembly);
            workerDrone.PickUpRobotPart(randomWeapon);

            workerDrone.TransportRobotParts();
            Assert.True(workerDrone.GetTransportationMechanisms().Any());

            var transportMechanism = workerDrone.GetTransportationMechanisms().First();
            Assert.IsInstanceOf<DeliveryBayTransportMechanism>(transportMechanism);
        }

Java

    public void workerDroneReturnsToDeliveryBayAfterDeliveringRobotParts() {
        workerDrone workerDrone = new mockedWorkerDrone();

        robotPart randomAssembly = new mockedAssembly();
        robotPart randomWeapon = new mockedWeapon();

        workerDrone.pickUpRobotPart(randomAssembly);
        workerDrone.pickUpRobotPart(randomWeapon);

        workerDrone.transportRobotParts();

        Iterator<transportMechanism> iterator = workerDrone.getTransportMechanisms().iterator();
        assertTrue(iterator.hasNext());

        transportMechanism transportMechanism = iterator.next();
        assertThat(transportMechanism, instanceOf(deliveryBayTransportMechanism.class));
    }

Notice here that we assert that our WorkerDrone contains a single DeliveryBayTransportMechanism. This implementation, when executed, simply returns the WorkerDrone to the DeliveryBay. Let’s look at how this works:

C#

        public void TransportRobotParts() {
            foreach (var transportMechanism in _transportMechanisms) {
                transportMechanism.OffLoadRobotPart();
            }

            _transportMechanisms.Clear();

            var deliveryBayTransportMechanism = new DeliveryBayTransportMechanism();
            _transportMechanisms.Add(deliveryBayTransportMechanism);

            deliveryBayTransportMechanism.EnterRoom();
        }

Java

    public void transportRobotParts() {
        for (Iterator<transportMechanism> i = _transportMechanisms.iterator(); i.hasNext(); ) {
            transportMechanism transportMechanism = i.next();
            transportMechanism.offLoadRobotPart();
        }

        _transportMechanisms.clear();

        deliveryBayTransportMechanism deliveryBayTransportMechanism = new deliveryBayTransportMechanism();
        _transportMechanisms.add(deliveryBayTransportMechanism);

        deliveryBayTransportMechanism.enterRoom();
    }

Note that we explicitly create a DeliveryBayTransportMechanism instance and execute the enterRoom method. Now our WorkerDrone has returned to DeliveryBay after successful delivery of its payload. We’ve demonstrated how TDD and OOD can facilitate change during the software development life-cycle.

The next tutorial in the series will focus on building Robots.

Connect with me:

RSSGitHubTwitter
LinkedInYouTubeGoogle+

Object Oriented, Test Driven Design in C# and Java: A Practical Example Part #3

Download the code in C#
Download the code in Java

Check out my interview on .NET Rocks! – TDD on .NET and Java with Paul Mooney

For a brief overview, please refer to this post.

We’ve provided our WorkerDrones with a means to determine an appropriate method of transportation by inspecting any given RobotPart implementation. Now WorkerDrones may select a TransportationMechanism implementation that suits each RobotPart. But we have yet to implement the actual logic involved. This is what we’ll cover in this tutorial. Look at how eager the little guy is! Let’s not delay; he’s got plenty of work to do.

WorkerDrone

Once again, here is our narrative:

“Mechs with Big Guns” is a factory that produces large, robotic vehicles designed to shoot other large, robotic vehicles. Robots are composed of several robotic parts, delivered by suppliers. Parts are loaded into a delivery bay, and are transported by worker drones to various rooms; functional parts such as arms, legs, etc., are dispatched to an assembly room. Guns and explosives are dispatched to an armoury.
The factory hosts many worker drones to assemble the robots. These drones will specialise in the construction of 1 specific robot, and will require all parts that make up that robot in order to build it. Once the drone has acquired all parts, it will enter the assembly room and build the robot. Newly built robots are transported to the armoury where waiting drones outfit them with guns. From time to time, two robots will randomly be selected from the armoury, and will engage one another in the arena, an advanced testing-ground in the factory. The winning robot will be repaired in the arena by repair drones. Its design will be promoted on a leader board, tracking each design and their associated victories.

Let’s look at what exactly happens when we transport a RobotPart.
First, the WorkerDrone needs to identify the RobotPart that it just picked up, so that it can transport the part to the correct FactoryRoom. Let’s dive right in.

In the previous tutorial, we defined a means to do this by examining a RobotPart's RobotPartCategory and returning an appropriate TransportMechanism. Now, let’s add logic to our TransportMechanism.

First, we need to keep track of the FactoryRoom where we’ll offload the RobotParts:

C#

 private FactoryRoom _factoryRoom;

Java

    private E _factoryRoom;

First of all, what can we tell about the difference between both implementations? Both contain private properties, but our C# implementation is explicitly bound to a FactoryRoom object. Our Java implementation, on the other hand, seems to be bound to the letter “E”.

“What’s that all about?”

The difference in implementations can be explained by discussing Generics. Essentially, Generics allow us to define an action, like a method, without defining a concrete return-type or input parameter – instead, we define these in concrete implementations of our abstraction. At this point, rather than go off-topic, I’ll provide a link to a thorough tutorial on this subject in C#.

In a nutshell, the difference in implementations comes down to a personal preference – I prefer Java’s implementation of Generics over C#’s, specifically Java’s support for covariance and contravariance. Again, I’m happy to follow up with this offline, or to host a separate post on the subject, but for now, let’s keep going.

Let’s look at our Java implementation of transportMechanism:

public abstract class transportMechanism&lt;E extends factoryRoom, U extends robotPart&gt; {

Here, we’re telling the compiler that our transportMechanism class will require 2 concrete implementations, both of which should be derived from factoryRoom and robotPart respectively. To illustrate this, let’s look at armouryTransportMechanism, a class derived from transportMechanism in Java:

public class armouryTransportMechanism extends transportMechanism&lt;armoury, weapon&gt; {

    @Override
    public armoury getFactoryRoom() {
        return new armoury();
    }
}

Notice our Generic implementation of factoryRoom and robotPart map to armoury and weapon, respectfully.

I’ll cover more about Generics on request. For now, let’s co back to our design.

Our TransportMechanism needs to return an appropriate FactoryRoom:

C#

public abstract FactoryRoom GetFactoryRoom();

Java

public abstract E getFactoryRoom();

So, what actually happens when a WorkerDrone moves RobotParts to a FactoryRoom? The WorkerDrone needs to enter the FactoryRoom, and then offload its components into the FactoryRoom:

C#

        public void EnterRoom() {
            _factoryRoom = GetFactoryRoom();
            _factoryRoom.AddTransportationMechanism(this);
        }

        public FactoryRoom OffLoadRobotParts(List&lt;RobotPart&gt; robotParts) {
            if (_factoryRoom == null) {
                EnterRoom();
            }
            _factoryRoom.SetRobotParts(new List&lt;RobotPart&gt;(robotParts));
            robotParts.Clear();

            return _factoryRoom;
        }

Java

    public void enterRoom() {
        _factoryRoom = getFactoryRoom();
        _factoryRoom.addTransportationMechanism(this);
    }

    public E offLoadRobotParts(List&lt;U&gt;robotParts) {
        if (_factoryRoom == null) {
            enterRoom();
        }
        _factoryRoom.setRobotParts(new ArrayList&lt;U&gt;(robotParts));
        robotParts.clear();

        return _factoryRoom;
    }

Here is a breakdown of what’s happening:

Our TransportMechanism returns a FactoryRoom implementation, based on the RobotPart carried by the WorkerDrone, and then the FactoryRoom adds the TransportationMechanism to its list of occupants:

C#

        public void AddTransportationMechanism(TransportMechanism transportMechanism) {
            _transportMechanisms.Add(transportMechanism);
        }

Java

    public void addTransportationMechanism(transportMechanism transportMechanism) {
        _transportMechanisms.add(transportMechanism);
    }

OK. Now our WorkerDrone has entered the FactoryRoom. It should now offload its RobotParts via the OffLoadRobotParts method above. Here’s what’s happening:

  • A safeguard is in place to ensure that the WorkerDrone enters the room before offloading components
  • The WorkerDrones RobotPart payload is copied to the FactoryRoom
  • The WorkerDrones RobotPart payload is emptied

“Why the safeguard? Can’t we just explicitly call the EnterRoom method before calling OffLoadRobotParts?”

Yes, but let’s offer another layer of protection for consuming applications. After all, if a developer forgot to ensure that a WorkerDrone enters a room before offloading RobotParts, the system would crash. Even if we implemented counter-measures to prevent this, our WorkerDrone would effectively dump its payload somewhere in the Factory.

What do you expect me to do now?!?

What do you expect me to do now?!?

Our WorkerDrone is now housed within an appropriate FactoryRoom, and has offloaded its RobotParts to that FactoryRoom.

“So how did we get here?”

Let’s examine the associated Unit Test:

C#

        [Test]
        public void WorkerDroneOffLoadsRobotParts() {
            WorkerDrone workerDrone = new MockedWorkerDrone();
            RobotPart robotPart = new MockedRobotPart(RobotPartCategory.Assembly);

            workerDrone.PickUpRobotPart(robotPart);
            var factoryRoom = workerDrone.TransportRobotParts();

            Assert.AreEqual(0, workerDrone.GetRobotPartCount());
            Assert.AreEqual(1, factoryRoom.GetRobotPartCount());
            Assert.IsInstanceOf&lt;AssemblyRoom&gt;(factoryRoom);

            robotPart = new MockedRobotPart(RobotPartCategory.Weapon);

            workerDrone.PickUpRobotPart(robotPart);
            factoryRoom = workerDrone.TransportRobotParts();

            Assert.AreEqual(0, workerDrone.GetRobotPartCount());
            Assert.AreEqual(1, factoryRoom.GetRobotPartCount());
            Assert.IsInstanceOf&lt;Armoury&gt;(factoryRoom);
        }

Java

    @Test
    public void workerDroneOffLoadsRobotParts() {
        workerDrone workerDrone = new mockedWorkerDrone();
        robotPart robotPart = new mockedRobotPart(robotPartCategory.assembly);

        workerDrone.pickUpRobotPart(robotPart);
        factoryRoom factoryRoom = workerDrone.transportRobotParts();

        assertEquals(0, workerDrone.getRobotPartCount());
        assertEquals(1, factoryRoom.getRobotPartCount());
        assertThat(factoryRoom, instanceOf(assemblyRoom.class));

        robotPart = new mockedRobotPart(robotPartCategory.weapon);

        workerDrone.pickUpRobotPart(robotPart);
        factoryRoom = workerDrone.transportRobotParts();

        assertEquals(0, workerDrone.getRobotPartCount());
        assertEquals(1, factoryRoom.getRobotPartCount());
        assertThat(factoryRoom, instanceOf(armoury.class));
    }

Notice out first pair of Asserts. We’ve transported the RobotParts from WorkerDrone to FactoryRoom, and simply assert that both components contain the correct number of RobotParts. Next, we assert that our TransportMechanism has selected the correct FactoryRoom instance; Weapons go to the Armoury, Assemblies to the AssemblyRoom.

“Great. I just looked at those FactoryRoom and RobotPart implementations. They’re all implementations of abstractions. Why didn’t you use interfaces, instead of abstract classes?”

There are 2 reasons for this:

  1. Our abstractions contain methods that need to be accessed by the implementations
  2. Our implementations are instances of our abstractions from a real-world perspective – they don’t just exhibit a set of behaviours.

It’s worth noting that a class can derive from a single class only in both C# and Java, whereas a class can derive from many interfaces as you like.

The next tutorial in the series will focus on returning our WorkerDrones to the DeliveryBay, and outlining the structure of RobotBuilders.

Connect with me:

RSSGitHubTwitter
LinkedInYouTubeGoogle+

Object Oriented, Test Driven Design in C# and Java: A Practical Example Part #2

Download the code in C#
Download the code in Java

Check out my interview on .NET Rocks! – TDD on .NET and Java with Paul Mooney

For a brief overview, please refer to this post.

Following on from the previous tutorial, where we looked at Supplier and DeliveryBay objects, let’s move on to WorkerDrones.

Once again, here is our narrative:

“Mechs with Big Guns” is a factory that produces large, robotic vehicles designed to shoot other large, robotic vehicles. Robots are composed of several robotic parts, delivered by suppliers. Parts are loaded into a delivery bay, and are transported by worker drones to various rooms; functional parts such as arms, legs, etc., are dispatched to an assembly room. Guns and explosives are dispatched to an armoury.
The factory hosts many worker drones to assemble the robots. These drones will specialise in the construction of 1 specific robot, and will require all parts that make up that robot in order to build it. Once the drone has acquired all parts, it will enter the assembly room and build the robot. Newly built robots are transported to the armoury where waiting drones outfit them with guns. From time to time, two robots will randomly be selected from the armoury, and will engage one another in the arena, an advanced testing-ground in the factory. The winning robot will be repaired in the arena by repair drones. Its design will be promoted on a leader board, tracking each design and their associated victories.

Worker Drones

Worker Drones have several functions, as per the narrative, so we’re going to tackle the implementation one step at a time. It seems that at a fundamental level, the drones need to be able to differentiate between different RobotParts. For example, consider a situation in which the following set of robotic legs arrive in the DeliveryBay:

Walker Robot, Vertical

In that case, our WorkerDrone needs to transport these to the AssemblyRoom. However, check out the guns on this guy!!!

White Mech weapon on a white background

Those weapons, should they arrive in the DeliveryBay, need to be transported to the Armoury.

Essentially, WorkerDrone behaviour depends on the category that each RobotPart is associated with, and can change at runtime. Consider the following scenario:

  1. WorkerDrone enters DeliveryBay and collects a RobotPart
  2. The RobotPart is identified as an item that belongs in the AssemblyRoom
  3. WorkerDrone transports the RobotPart to the AssemblyRoom
  4. WorkerDrone picks up the next RobotPart, which happens to be a Weapon
  5. WorkerDrone delivers this to the Armoury

WorkerDrones have 2 separate types of behaviour that govern RobotPart transportation. This is commonly known as the Strategy Design Pattern.

“Wait, that actually sounds like a Bridge pattern to me!”

Structurally, they are both the same. The difference is in their implementation. A Bridge is a behaviour that is set at design-time, and remains constant. A Strategy on the other hand, can change at runtime.

What we’re essentially doing is abstracting WorkerDrone transportation behaviour to separate classes that share the same abstraction. Our WorkerDrone simply chooses the correct implementation to suit a given scenario, and the implementation does the work.

Let’s keep thiings simple for the moment; we won’t actually implement the act of transporting RobotParts, we’ll simply implement the fundamental components involved.

To start, we need to identify RobotParts. So let’s tag them with a specific category – either Assembly, or Weapon. First, we need to define these categories:

C#

    public enum RobotPartCategory {
        Assembly,
        Weapon
    }

Java

public enum robotPartCategory {
    assembly,
    weapon
}

Now, we modify our RobotPart astraction to accept either of these categories:

C#

    public abstract class RobotPart {
        private readonly RobotPartCategory _robotPartCategory;

        public RobotPartCategory RobotPartCategory { get { return _robotPartCategory; } }

        protected RobotPart(RobotPartCategory robotPartCategory) {
            _robotPartCategory = robotPartCategory;
        }
    }

Java

public abstract class robotPart {
    protected robotPartCategory robotPartCategory;

    public robotPartCategory getRobotPartCategory() {
        return robotPartCategory;
    }

    protected robotPart(robotPartCategory robotPartCategory) {
        this.robotPartCategory = robotPartCategory;
    }
}

So far, so good. We spoke about different transportation behaviours. So let’s first create the abstraction:

C#

    public abstract class TransportMechanism {}

Java

public abstract class transportMechanism {}

And both implementations; Assembly and Weapon. Remember: we’re not implementing any logic in these classes yet – we first need to ensure that our WorkerDrone can apply the correct behaviour based on each RobotPart:

C#

    public class AssemblyRoomTransportMechanism : TransportMechanism {}
    public class ArmouryTransportMechanism : TransportMechanism {}

Java

public class assemblyRoomTransportMechanism extends transportMechanism {}
public class armouryTransportMechanism extends transportMechanism {}

Now let’s implement a simple method that allows our WorkerDrone to choose the manner in which it will transport a given RobotPart, by first examining that RobotPart

C#

    public abstract class WorkerDrone {
        public TransportMechanism TransportMechanism { get; private set; }

        public void IdentifyRobotPart(RobotPart robotPart) {
            switch (robotPart.RobotPartCategory) {
                case RobotPartCategory.Assembly:
                    TransportMechanism = new AssemblyRoomTransportMechanism();
                    break;
                case RobotPartCategory.Weapon:
                    TransportMechanism = new ArmouryTransportMechanism();
                    break;
            }
        }
    }

Java

public class workerDrone {
    private transportMechanism _transportMechanism;

    public transportMechanism getTransportMechanism() {
        return _transportMechanism;
    }

    public void identifyRobotPart(robotPart robotPart) {
        switch (robotPart.getRobotPartCategory()) {
            case assembly:
                _transportMechanism = new assemblyRoomTransportMechanism();
                break;
            case weapon:
                _transportMechanism = new armouryTransportMechanism();
                break;
        }
    }
}

As usual, our WorkerDrone is an abstraction and we apply a mocked implementation in our associated unit test:

C#

    public void WorkerDroneIdentifiesRobotPart() {
            var robotPart = new MockedRobotPart(RobotPartCategory.Assembly);
            var workerDrone = new MockedWorkerDrone();

            workerDrone.IdentifyRobotPart(robotPart);
            Assert.IsInstanceOf<AssemblyRoomTransportMechanism>(workerDrone.TransportMechanism);

            robotPart = new MockedRobotPart(RobotPartCategory.Weapon);

            workerDrone.IdentifyRobotPart(robotPart);
            Assert.IsInstanceOf<ArmouryTransportMechanism>(workerDrone.TransportMechanism);
        }

Java

    public void workerDroneIdentifiesRobotPart() {
        robotPart robotPart = new mockedRobotPart(robotPartCategory.assembly);
        workerDrone workerDrone = new mockedWorkerDrone();

        workerDrone.identifyRobotPart(robotPart);
        assertThat(workerDrone.getTransportMechanism(), instanceOf(assemblyRoomTransportMechanism.class));

        robotPart = new mockedRobotPart(robotPartCategory.weapon);

        workerDrone.identifyRobotPart(robotPart);
        assertThat(workerDrone.getTransportMechanism(), instanceOf(armouryTransportMechanism.class));
    }

Let’s walk through this test:

  1. We instnantiate a new RobotPart as an Assembly
  2. We instatiate our WorkerDrone
  3. The WorkerDrone examines the RobotPart and instantiates its TransportationMechanism appropriately
  4. We assert that the correct TransportationMechanism behavioural implementation has been applied for the Assembly RobotPart
  5. We repeat the process for a Weapon RobotPart

In the next tutorial, we’ll implement the logic involved in transporting RobotParts from the DeliveryRoom to both AssemblyRoom and Armoury.

Connect with me:

RSSGitHubTwitter
LinkedInYouTubeGoogle+