How to Create a Simple Circuit Breaker in C#

Posted on: 2015-12-01

This article demonstrates how to create a simple Circuit Breaker in C#. It’s under 180 lines of code. Before going into detail, let’s take one paragraph to explain the circuit breaker pattern.

Circuit Breaker pattern is named from house circuit breaker — something fail, it opens the circuit, thus does not do any damage. With code, the circuit breaker pattern is useful when we access a resource that can slow down the system. For example, you have a website that connect different services on different servers, and one server goes down. You do not want subsequent calls to that server to call all the time that server. Mostly because it will slow down the whole website by having to timeout before being able to continue the code. This is a simple example, but this can be applied to other scenarios. The principle remains the same: execute some code, if it works nothing change; if it fails than we put the system into a close state. The open state allows to skip trying to execute the failed code and do something else until the circuit goes into an alternate state where it tries again to see if it can close the circuit again.

The circuit breaker demonstrates, here, does not rely on a timer. A lot of time, this pattern is closed for an amount of time before trying to re-open. However, I do not like having a timer running so I simply prefer to keep the date time and check when attempting to call. I also like using the state pattern for this circuit breaker pattern since we are moving through 3 different states (opened, half opened, closed).

The first class is the one that each of the three classes will inherit. It contains some virtual methods that the state can define if required. It also has a constructor that take the circuit breaker class to let you access some methods like increasing the default count when an exception is rose.

public abstract class CircuitBreakerState { 
  protected readonly CircuitBreaker circuitBreaker;

  protected CircuitBreakerState(CircuitBreaker circuitBreaker) { 
    this.circuitBreaker = circuitBreaker;
  }

  public virtual CircuitBreaker ProtectedCodeIsAboutToBeCalled() { 
    return this.circuitBreaker; 
  } 
  
  public virtual void ProtectedCodeHasBeenCalled() { } 
  
  public virtual void ActUponException(Exception e) { 
    circuitBreaker.IncreaseFailureCount(); 
  }

  public virtual CircuitBreakerState Update() { return this; } 
} 

The first state is the open state. It contains the time where the state open (where something went wrong). When the state is instantiated, it set the current date time. This is required because if we want to update the state to the next one, the half-open one, we need to have at least a down period defined by the circuit breaker. It overrides two methods. The ProtectedCodeIsAboutToBeCalled where it tries to call the protected code. By protected code, I mean the code that is under the circuit breaker. This method try to update the state. If it can, than it goes into the half open state. Otherwise, it remains open.

public class OpenState : CircuitBreakerState { 
  private readonly DateTime openDateTime; 
  public OpenState(CircuitBreaker circuitBreaker) : base(circuitBreaker) { 
    openDateTime = DateTime.UtcNow; 
  }

  public override CircuitBreaker ProtectedCodeIsAboutToBeCalled() { 
    base.ProtectedCodeIsAboutToBeCalled(); 
    this.Update(); 
    return base.circuitBreaker; 
  }

  public override CircuitBreakerState Update() { 
    base.Update(); 
    if (DateTime.UtcNow >= openDateTime + base.circuitBreaker.Timeout) { 
      return circuitBreaker.MoveToHalfOpenState(); 
    } 
    return this; 
  } 
} 

The second state is half open. This state is active only when the circuit went into something wrong and it tries to come back into an open one. This one is pretty small. It goes into open state if something is wrong, or it goes into the closed state if no exception is thrown. So, this is called after the threshold of time is done, it tries to close the circuit by calling the protected code. If this one fail, it creates a new open state where the time is resetted.

public class HalfOpenState : CircuitBreakerState { 
  public HalfOpenState(CircuitBreaker circuitBreaker) : base(circuitBreaker) { 

  }

  public override void ActUponException(Exception e) { 
    base.ActUponException(e); 
    circuitBreaker.MoveToOpenState(); 
  }

  public override void ProtectedCodeHasBeenCalled() { 
    base.ProtectedCodeHasBeenCalled(); 
    circuitBreaker.MoveToClosedState(); 
  } 
} 

The last state is the closed state. This is the first state you start the circuit pattern and the one you want to be. This is where we set the failure count and move to open when the failure count reach the threshold is met. This is not absolutely required, but it lets you have some window before opening the circuit. For example, sometime the connection can just be not reliable and retrying right after make it works. So you can set a failure threshold of 3 failures before going in an open state for 5 minutes.

public class ClosedState : CircuitBreakerState { 
  public ClosedState(CircuitBreaker circuitBreaker) : base(circuitBreaker) {
    circuitBreaker.ResetFailureCount();
  }

  public override void ActUponException(Exception e) { 
    base.ActUponException(e); 
    if (circuitBreaker.IsThresholdReached()) { 
      circuitBreaker.MoveToOpenState(); 
    } 
  } 
} 

The biggest piece of code is inside the main class : the CircuitBreaker. This class contains the threshold for failure, the time to wait when the circuit is open and also all the locking mechanism.

public class CircuitBreaker { 
  private readonly object monitor = new object(); 
  private CircuitBreakerState state;

  public CircuitBreaker(int threshold, TimeSpan timeout) { 
    if (threshold < 1) { 
      throw new ArgumentOutOfRangeException("threshold", "Threshold should be greater than 0"); 
    }

    if (timeout.TotalMilliseconds < 1) { 
      throw new ArgumentOutOfRangeException("timeout", "Timeout should be greater than 0");
     }

    Threshold = threshold; 
    Timeout = timeout;
     MoveToClosedState(); 
  }

  public int Failures { get; private set; } 
  public int Threshold { get; private set; } 
  public TimeSpan Timeout { get; private set; } 
  public bool IsClosed { get { return state.Update() is ClosedState; } }
  public bool IsOpen { get { return state.Update() is OpenState; } }
  public bool IsHalfOpen { get { return state.Update() is HalfOpenState; } }

  internal CircuitBreakerState MoveToClosedState() { 
    state = new ClosedState(this); 
    return state;
  }

  internal CircuitBreakerState MoveToOpenState() { 
    state = new OpenState(this); 
    return state; 
  }

  internal CircuitBreakerState MoveToHalfOpenState() { 
    state = new HalfOpenState(this); 
    return state; 
  }

  internal void IncreaseFailureCount() { 
    Failures++; 
  }

  internal void ResetFailureCount() { 
    Failures = 0; 
  }

  public bool IsThresholdReached() { 
    return Failures >= Threshold; 
  }

  private Exception exceptionFromLastAttemptCall = null;

  public Exception GetExceptionFromLastAttemptCall() { 
    return exceptionFromLastAttemptCall; 
  }

  public CircuitBreaker AttemptCall(Action protectedCode) { 
    this.exceptionFromLastAttemptCall = null; 
    lock(monitor) { 
      state.ProtectedCodeIsAboutToBeCalled(); 
      if (state is OpenState) { 
        return this; // Stop execution of this method 
      } 
    }

    try { 
      protectedCode(); 
    } catch (Exception e) { 
      this.exceptionFromLastAttemptCall = e; 
      lock(monitor) { 
        state.ActUponException(e); 
      } 
      return this; // Stop execution of this method 
    }

    lock (monitor) { 
      state.ProtectedCodeHasBeenCalled(); 
    } 
    return this; 
  }

  public void Close() { 
    lock (monitor) { 
      MoveToClosedState();
    } 
  }

  public void Open() { 
    lock (monitor) {
      MoveToOpenState(); 
    } 
  } 
} 

This is easier to use than you may think. You mostly just need to use AttemptCall. You need to instantiate the circuit breaker. This must be a singleton since we want to keep the count of failure and the time when the circuit is open.

private static readonly CircuitBreaker circuitBreaker = new CircuitBreaker(3, TimeSpan.FromMinutes(15)); 

Then, we need to call the circuit breaker for the code that is susceptible to thrown an exception, to fail.

myCircuitBreaker.AttemptCall(()=>{yourCode();}).IsClosed?"AllFine":"Something wrong"; 

Of course, the example above is not really a normal situation -- you wouldn't just return a string. In real situation, you AttemptCall and get back the CircuitBreaker. Then, you have a switch case with the different state and act accordingly. For example, I usually on the IsClosed state get some log to try to improve the protected code if this one is getter there to often.

To conclude, this pattern is really easy to implement, easy to use and can help you in many situation instead of failing or having to repeat logic on failure code. You can use this pattern if you are having a caching system. If this one is down, you can go in the database. You can also use this pattern for third party API. If this one fail, you can notify the user that the system is temporary down without hammering the third party or you can use not fresh data for few minutes. You also have a lot of liberty about how to use this pattern. You can setup that it fails very fast with only a single try or being close for few milliseconds or few minutes depending of how reliable it the protected code. If you are interested in the code, the whole circuit breaker code is open source on GitHub. You can also use this one by using this Circuit Breaker Nuget package.

Check out the video Webucator made on my article: