5
\$\begingroup\$

I wanted to be able to run a series of Python functions and capture every exception that happens. The solution uses a AggregateException class and tries to leverage Python scoping in order to aggregate a sequence of Exceptions raised.

The Solution


class AggregateException(Exception):

    def __init__(self, errors: List[Exception], error_count: int = 0) -> None:
        self.errors = errors
        self.error_count = error_count

    @classmethod
    def aggregate(cls, *steps: Callable[[], None]) -> None:
        errors: List[Exception] = []
        for step in steps:
            try:
                step()
            except Exception as e:
                errors.append(e)

        if errors:
            raise cls(errors=errors, error_count=len(errors))

Usage

from typing import Callable, List


class DomainSpecificException(Exception):
    ...


class AggregateException(Exception):

    def __init__(self, errors: List[DomainSpecificException], error_count: int = 0) -> None:
        self.errors = errors
        self.error_count = error_count

    @classmethod
    def aggregate(cls, *steps: Callable[[], None]) -> None:
        errors: List[Exception] = []
        for step in steps:
            try:
                step()
            except Exception as e:
                errors.append(e)

        if errors:
            raise cls(errors=errors, error_count=len(errors))


def main():

    def foo():
        # stuffz that could go wrong...
        raise DomainSpecificException("Something went wrong")

    def bar():
        # more stuffz that could go wrong...
        raise DomainSpecificException("Oh no!")

    try:
        AggregateException.aggregate(foo, bar)
    except AggregateException as err:
        print(err.errors)


if __name__ == "__main__":
    main()

Output

[DomainSpecificException('Something went wrong'), DomainSpecificException('Oh no!')]
\$\endgroup\$
2
  • 1
    \$\begingroup\$ Are you using Python 3.11? \$\endgroup\$
    – Peilonrayz
    Jun 16, 2023 at 17:49
  • \$\begingroup\$ No, but not a huge job for me to bump up so feel free to show a solution targeting 3.11. \$\endgroup\$
    – Bob
    Jun 17, 2023 at 7:18

2 Answers 2

5
\$\begingroup\$

Some simplification is called-for.

Drop the idea of an AggregateException that throws the collection of exceptions, and instead make a context manager ExceptionAggregator.

You have domain exceptions - good! Only pay attention to those and not all Exceptions.

Don't keep an error_count; that's redundant.

Write unit tests.

The following no longer needs callables, though - if your callables have any significant amount of code in them - you should keep the functions you've defined and just call them from context management scope.

Suggested

from types import TracebackType
from typing import Optional, Type


class DomainSpecificException(Exception):
    pass


class ExceptionAggregator:
    def __init__(self) -> None:
        self.errors: list[DomainSpecificException] = []

    def __enter__(self) -> None:
        pass

    def __exit__(
        self,
        exc_type: Optional[Type[Exception]],
        exc_val: Optional[Exception],
        exc_tb: Optional[TracebackType],
    ) -> bool:
        if exc_type is DomainSpecificException:
            self.errors.append(exc_val)
            return True
        return False


def test():
    agg = ExceptionAggregator()
    with agg:
        pass
    assert agg.errors == []

    try:
        with agg:
            raise ValueError("Something went more wrong")
        raise AssertionError()
    except ValueError:
        pass
    assert agg.errors == []

    with agg:
        raise DomainSpecificException("Something went wrong")

    assert len(agg.errors) == 1
    assert isinstance(agg.errors[0], DomainSpecificException)


if __name__ == "__main__":
    test()
\$\endgroup\$
5
  • 2
    \$\begingroup\$ Have __enter__() return self. Then the code can be with ExceptionAggregator() as agg: \$\endgroup\$
    – RootTwo
    Jun 16, 2023 at 20:53
  • 1
    \$\begingroup\$ @RootTwo ...why? It doesn't actually need to be referenced in context. \$\endgroup\$
    – Reinderien
    Jun 16, 2023 at 21:07
  • 1
    \$\begingroup\$ ...why not? On the one hand, it's a very common way to use a context manager (open, decimal, etc.). On the other hand, this use case seems analogous to contextlib.supress which doesn't return anything from __enter__(). On the 3rd hand, warnings.catch_warnings() returns a list if you are logging the caught warnings and returns None if you aren't. \$\endgroup\$
    – RootTwo
    Jun 17, 2023 at 1:05
  • 1
    \$\begingroup\$ Think about the use case: this is explicitly for stringing together multiple (not one) blocks that are independent from each other, and the results aren't useful until after the block has exited (whether by exception or not). An API that gives the developer an object they should never use is not a good idea. Moreover, the with EA() as foo syntax used multiple times would decorrelate any exceptions into separate lists, which is the opposite of what this thing was meant to do. \$\endgroup\$
    – Reinderien
    Jun 17, 2023 at 1:20
  • 1
    \$\begingroup\$ Be aware that tests written like this might silently pass in optimised mode. That's why the unittest module has all those .assertTrue methods. \$\endgroup\$
    – wizzwizz4
    Jun 17, 2023 at 14:08
4
\$\begingroup\$

The main question is why and where would you need such functionality?

Consider these shortcomings:

  • you can only run functions that don't take any arguments
  • you can't use cycles / conditional logic / modify the order of execution in any way
  • you catch every single exception which is never a good thing (e.g. you won't be able to Ctrl+C out of your program while this is running)

There is another problem:

The entire point of raising exceptions is breaking the control flow in the middle of execution. aggregate function raises its exception at the very end of it. At this point you can just return aggregated exceptions as a list.

This syntax: AggregateException.aggregate(...) is not intuitive as it hides execution of multiple functions under an AggregateExecution method. Taking the last point into account we can get rid of the exception and write it like so:

def aggregate_exceptions(blocks):
    exceptions = []
    for block in blocks:
        try:
            block()
        except Exception as e:
            exceptions.append(e)
    return exceptions

Though the main point still stands, what are the usecases?

With that said, the code itself is very clean, keep it up 👍

\$\endgroup\$
5
  • 3
    \$\begingroup\$ The general point about exceptions stands, but the specific point about not being able to Ctrl+C is incorrect. KeyboardInterrupt does not inherit from Exception. \$\endgroup\$
    – Reinderien
    Jun 16, 2023 at 12:33
  • \$\begingroup\$ @Reinderien oops my bad \$\endgroup\$ Jun 16, 2023 at 13:22
  • \$\begingroup\$ I find your answer to be unnecessary dismissive. Ignoring PEP 654. I have a file with data in multiple formats. I run validators over the file to ensure all the wanted fields are in the file, and then post validation build the objects. The validation step can run into a number of issues, each which following your advice would only raise one exception. Following your advice fixing a broken file would be an annoying cycle of run error, run error... In short I don't think being judgemental improves your answer. \$\endgroup\$
    – Peilonrayz
    Jun 19, 2023 at 0:44
  • \$\begingroup\$ @Peilonrayz "fixing a broken file would be an annoying cycle of run error, run error" not sure what this means, can you elaborate? As for being judgemental, look at the OP's example. Do we really need an extra class for that? We don't need an MVVM / MVI architecture for a calculator app, but it does't mean those architectures are bad. I haven't seen a good use for those AggregateExceptions yet, but I won't say there aren't any. Anyway this is not the one. \$\endgroup\$ Jun 19, 2023 at 9:51
  • \$\begingroup\$ @QuasiStellar "look at the OP's example. Do we really need an extra class for that?" The example I gave would fit nicely with the code the OP provided. "I haven't seen a good use for those AggregateExceptions" would explain why your answer is distasteful. \$\endgroup\$
    – Peilonrayz
    Jun 19, 2023 at 16:07

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.