While iterating the context (once), multiple sources can be dispatched.
So if we get a timeout and shortly after the weak-ref callback, then
we still need to honor the weak-ref.
In particular, because weak_ref.unref() is not safe to do after
the object was already destroyed. So we need to be correct about
tracking destruction.