How to Create a Simple Circuit Breaker in C#

Posted on: December 1, 2015

This article demonstrates how to create a simple Circuit Breaker in C#. It’s under 180 lines of code. Before going into detail, let’s take one paragraph to explain the circuit breaker pattern.

Circuit Breaker pattern is named from house circuit breaker — something fail, it opens the circuit, thus does not do any damage. With code, the circuit breaker pattern is useful when we access a resource that can slow down the system. For example, you have a website that connect different services on different servers, and one server goes down. You do not want subsequent calls to that server to call all the time that server. Mostly because it will slow down the whole website by having to timeout before being able to continue the code. This is a simple example, but this can be applied to other scenarios. The principle remains the same: execute some code, if it works nothing change; if it fails than we put the system into a close state. The open state allows to skip trying to execute the failed code and do something else until the circuit goes into an alternate state where it tries again to see if it can close the circuit again.

The circuit breaker demonstrates, here, does not rely on a timer. A lot of time, this pattern is closed for an amount of time before trying to re-open. However, I do not like having a timer running so I simply prefer to keep the date time and check when attempting to call. I also like using the state pattern for this circuit breaker pattern since we are moving through 3 different states (opened, half opened, closed).

The first class is the one that each of the three classes will inherit. It contains some virtual methods that the state can define if required. It also has a constructor that take the circuit breaker class to let you access some methods like increasing the default count when an exception is rose.

1public abstract class CircuitBreakerState { protected readonly CircuitBreaker circuitBreaker;
2
3protected CircuitBreakerState(CircuitBreaker circuitBreaker) { this.circuitBreaker = circuitBreaker; }
4
5public virtual CircuitBreaker ProtectedCodeIsAboutToBeCalled() { return this.circuitBreaker; } public virtual void ProtectedCodeHasBeenCalled() { } public virtual void ActUponException(Exception e) { circuitBreaker.IncreaseFailureCount(); }
6
7public virtual CircuitBreakerState Update() { return this; } }

The first state is the open state. It contains the time where the state open (where something went wrong). When the state is instantiated, it set the current date time. This is required because if we want to update the state to the next one, the half-open one, we need to have at least a down period defined by the circuit breaker. It overrides two methods. The ProtectedCodeIsAboutToBeCalled where it tries to call the protected code. By protected code, I mean the code that is under the circuit breaker. This method try to update the state. If it can, than it goes into the half open state. Otherwise, it remains open.

1public class OpenState : CircuitBreakerState { private readonly DateTime openDateTime; public OpenState(CircuitBreaker circuitBreaker) : base(circuitBreaker) { openDateTime = DateTime.UtcNow; }
2
3public override CircuitBreaker ProtectedCodeIsAboutToBeCalled() { base.ProtectedCodeIsAboutToBeCalled(); this.Update(); return base.circuitBreaker; }
4
5public override CircuitBreakerState Update() { base.Update(); if (DateTime.UtcNow >= openDateTime + base.circuitBreaker.Timeout) { return circuitBreaker.MoveToHalfOpenState(); } return this; } }

The second state is half open. This state is active only when the circuit went into something wrong and it tries to come back into an open one. This one is pretty small. It goes into open state if something is wrong, or it goes into the closed state if no exception is thrown. So, this is called after the threshold of time is done, it tries to close the circuit by calling the protected code. If this one fail, it creates a new open state where the time is resetted.

1public class HalfOpenState : CircuitBreakerState { public HalfOpenState(CircuitBreaker circuitBreaker) : base(circuitBreaker) { }
2
3public override void ActUponException(Exception e) { base.ActUponException(e); circuitBreaker.MoveToOpenState(); }
4
5public override void ProtectedCodeHasBeenCalled() { base.ProtectedCodeHasBeenCalled(); circuitBreaker.MoveToClosedState(); } }

The last state is the closed state. This is the first state you start the circuit pattern and the one you want to be. This is where we set the failure count and move to open when the failure count reach the threshold is met. This is not absolutely required, but it lets you have some window before opening the circuit. For example, sometime the connection can just be not reliable and retrying right after make it works. So you can set a failure threshold of 3 failures before going in an open state for 5 minutes.

1public class ClosedState : CircuitBreakerState { public ClosedState(CircuitBreaker circuitBreaker) : base(circuitBreaker) { circuitBreaker.ResetFailureCount(); }
2
3public override void ActUponException(Exception e) { base.ActUponException(e); if (circuitBreaker.IsThresholdReached()) { circuitBreaker.MoveToOpenState(); } } }

The biggest piece of code is inside the main class : the CircuitBreaker. This class contains the threshold for failure, the time to wait when the circuit is open and also all the locking mechanism.

1public class CircuitBreaker { private readonly object monitor = new object(); private CircuitBreakerState state;
2
3public CircuitBreaker(int threshold, TimeSpan timeout) { if (threshold < 1) { throw new ArgumentOutOfRangeException("threshold", "Threshold should be greater than 0"); }
4
5if (timeout.TotalMilliseconds < 1) { throw new ArgumentOutOfRangeException("timeout", "Timeout should be greater than 0"); }
6
7Threshold = threshold; Timeout = timeout; MoveToClosedState(); }
8
9public int Failures { get; private set; } public int Threshold { get; private set; } public TimeSpan Timeout { get; private set; } public bool IsClosed { get { return state.Update() is ClosedState; } }
10
11public bool IsOpen { get { return state.Update() is OpenState; } }
12
13public bool IsHalfOpen { get { return state.Update() is HalfOpenState; } }
14
15internal CircuitBreakerState MoveToClosedState() { state = new ClosedState(this); return state; }
16
17internal CircuitBreakerState MoveToOpenState() { state = new OpenState(this); return state; }
18
19internal CircuitBreakerState MoveToHalfOpenState() { state = new HalfOpenState(this); return state; }
20
21internal void IncreaseFailureCount() { Failures++; }
22
23internal void ResetFailureCount() { Failures = 0; }
24
25public bool IsThresholdReached() { return Failures >= Threshold; }
26
27private Exception exceptionFromLastAttemptCall = null;
28
29public Exception GetExceptionFromLastAttemptCall() { return exceptionFromLastAttemptCall; }
30
31public CircuitBreaker AttemptCall(Action protectedCode) { this.exceptionFromLastAttemptCall = null; lock(monitor) { state.ProtectedCodeIsAboutToBeCalled(); if (state is OpenState) { return this; // Stop execution of this method } }
32
33try { protectedCode(); } catch (Exception e) { this.exceptionFromLastAttemptCall = e; lock(monitor) { state.ActUponException(e); } return this; // Stop execution of this method }
34
35lock (monitor) { state.ProtectedCodeHasBeenCalled(); } return this; }
36
37public void Close() { lock (monitor) { MoveToClosedState(); } }
38
39public void Open() { lock (monitor) { MoveToOpenState(); } } }

This is easier to use than you may think. You mostly just need to use AttemptCall. You need to instantiate the circuit breaker. This must be a singleton since we want to keep the count of failure and the time when the circuit is open.

1private static readonly CircuitBreaker circuitBreaker = new CircuitBreaker(3, TimeSpan.FromMinutes(15));

Then, we need to call the circuit breaker for the code that is susceptible to thrown an exception, to fail.

1myCircuitBreaker.AttemptCall(()=>{yourCode();}).IsClosed?"AllFine":"Something wrong";

Of course, the example above is not really a normal situation -- you wouldn't just return a string. In real situation, you AttemptCall and get back the CircuitBreaker. Then, you have a switch case with the different state and act accordingly. For example, I usually on the IsClosed state get some log to try to improve the protected code if this one is getter there to often.

To conclude, this pattern is really easy to implement, easy to use and can help you in many situation instead of failing or having to repeat logic on failure code. You can use this pattern if you are having a caching system. If this one is down, you can go in the database. You can also use this pattern for third party API. If this one fail, you can notify the user that the system is temporary down without hammering the third party or you can use not fresh data for few minutes. You also have a lot of liberty about how to use this pattern. You can setup that it fails very fast with only a single try or being close for few milliseconds or few minutes depending of how reliable it the protected code. If you are interested in the code, the whole circuit breaker code is open source on GitHub. You can also use this one by using this Circuit Breaker Nuget package.

Check out the video Webucator made on my article: