So, how many times have you, while investigating an (admittedly rare ;)) production bug, stumbled upon an exception backtrace that just doesn’t have enough information? For example, an exception gets reraised, destroying the original backtrace with it. So now your logs say that your AI failed to take over the world with AIEpicFailError (great, you already know that), but not why (Timeout::Error while connecting to stackoverflow.com).
There are numerous suggestions online — one is to reraise the exception, but modify the exception first to preserve the original backtrace. That works, but you lose the original exception type, as well as the backtrace of this exception. Another is to log the backtrace before reraising at every point in the application — but this means that your logs now have multiple backtraces and it’s not obvious which ones are part of the same flow. None of these approaches sit well with me.
While we were having heated internal engineering arguments about how we could implement a chain of exceptions (e.g. with thread local storage — yuck), I found out that in fact, there is an Exception#cause method in Ruby since version 2.1! This means that something like this is possible:
class A < StandardError; end class B < StandardError; end the_exception = nil begin begin raise A rescue raise B
end rescue => exc the_exception = exc end the_exception # #<B: B> the_exception.cause # #<A: A> the_exception.cause.cause # nil
My GOD! This is amazing! This means that you can simply reraise an exception in a rescue block, and ruby automatically keeps a chain of what caused each preceding exception. Honestly, I have no idea why no one is talking about this, and why it took me so long to find it online.
So you get hyped, but then maybe you realize your application has multithreading. The multithreading library we use will execute a bunch of lambdas in a thread pool. You can then probe whether these lambdas were successful, or get the associated exception if there is one. However, this means you now have an array of exceptions. Is your beautiful Exception#cause approach doomed?
Fear not, for I have the solution!
class StandardErrorCollection < StandardError include Enumerable # Enumerable gives you a bunch of free methods (to_a, select, map...) as long as you implement each - awesome! attr_reader :causes def initialize(causes) @causes = causes end
def each(&block) @causes.each(&block) end end
This creates a StandardErrorCollection, which itself is a StandardError, meaning you can raise it. It is also Enumerable, which means you can iterate through it. With the above implementation, it will iterate through the causes of this exception. Here's example usage:
class A < StandardError; end the_exception = nil begin raise A rescue => exc the_exception = exc end
begin raise StandardErrorCollection.new(3.times.map { the_exception }) # pass an array of exceptions rescue => exc the_exception = exc end the_exception.causes # [#<A: A>, #<A: A>, #<A: A>]
Great! This resolves the multithreading issue — you now have an exception which was caused by multiple causes, and you can continue raising this single exception up the stack.
So now you have arrived at the top of your stack, and it’s time to log the backtraces. You have a single exception, which encompasses a chain of exceptions, with each one having either zero, one, or multiple causes. This sounds a lot like a tree to me! This means we can use a depth first search in order to find all the unique paths from root to leaf, and this gives us all the unique code execution paths. We can then log each path as a single log entry. So good, I might cry.
Here’s an implementation for a path traverser, which is built on top of a depth first search:
# this is a base class since there multiple types of traversal - path traversal, depth first search traversal, breadth first search traversal...
class ExceptionTraverser include Enumerable attr_reader :exception
def initialize(exception) @exception = exception end end class ExceptionPathTraverser < ExceptionTraverser def each(&block) return if @exception.nil? each_internal(@exception, [], &block) end private def each_internal(exception, path, &block) # NOTE: path.clone is required because otherwise to_a will return an empty array (since path will be an empty array after everything has executed) # NOTE: path.clone will clone the array, but not the exceptions, so duplicate exceptions will be the same object ID - awesome! return block.call(path.clone) if exception.nil? path.push(exception) if exception.respond_to?(:causes) exception.causes.each do |cause| each_internal(cause, path, &block) end else each_internal(exception.cause, path, &block) end path.pop end end
You will note that ExceptionTraverser is also Enumerable - this will allow us to enumerate through whatever it is we are traversing.
For ExceptionPathTraverser, we are traversing paths. To achieve this, the class simply implements the each method required for the object to be Enumerable, and yields the individual paths.
Example usage follows — we will be generating the following exception tree with the execution paths (E->F->C->B->A) and (E->F->D->C):
class A < StandardError; end class B < StandardError; end class C < StandardError; end class D < StandardError; end class E < StandardError; end class F < StandardErrorCollection; end def capture_exception captured_exception = nil begin yield rescue => exc captured_exception = exc end return captured_exception end
def reraise_with(exc) begin yield rescue raise exc end end
# pseudo execution of a multithreader which returns an array of exceptions exception_on_thread_one = capture_exception { reraise_with(C) { reraise_with(B) { raise A } } } exception_on_thread_two = capture_exception { reraise_with(D) { raise C } } all_thread_exceptions = [exception_on_thread_one, exception_on_thread_two]
final_exception = capture_exception { reraise_with(E) { raise F.new(all_thread_exceptions) } }
ExceptionPathTraverser.new(final_exception).each do |path| puts path.inspect end # [#<E: E>, #<F: F>, #<C: C>, #<B: B>, #<A: A>]
# [#<E: E>, #<F: F>, #<D: D>, #<C: C>]
You can now take each execution path, and log all the associated backtraces, as well as any additional information in a single log entry.
Hope you enjoyed this blog post, and find it useful in your endeavours!
Originally published at blog.cloud66.com on November 27, 2018.
