Hooking the Transaction Machinery

The transaction machinery allows application developers to register two different groups of callbacks to be called, one group before committing the transaction and one group after.

These hooks are not designed to be used as replacements for the two-phase commit machinery defined by a resource manager (see resourcemanager). In particular, hook functions must not raise or propagate exceptions.

Warning

Hook functions which do raise or propagate exceptions will leave the application in an undefined state.

The addBeforeCommitHook() Method

Let’s define a hook to call, and a way to see that it was called.

>>> log = []
>>> def reset_log():
...     del log[:]

>>> def hook(arg='no_arg', kw1='no_kw1', kw2='no_kw2'):
...     log.append("arg %r kw1 %r kw2 %r" % (arg, kw1, kw2))

Now register the hook with a transaction.

>>> from transaction import begin
>>> import transaction
>>> t = begin()
>>> t.addBeforeCommitHook(hook, ('1',))

We can see that the hook is indeed registered.

>>> [(hook.__name__, args, kws)
...  for hook, args, kws in t.getBeforeCommitHooks()]
[('hook', ('1',), {})]

When transaction commit starts, the hook is called, with its arguments.

>>> log
[]
>>> t.commit()
>>> log
["arg '1' kw1 'no_kw1' kw2 'no_kw2'"]
>>> reset_log()

A hook’s registration is consumed whenever the hook is called. Since the hook above was called, it’s no longer registered:

>>> from transaction import commit
>>> len(list(t.getBeforeCommitHooks()))
0
>>> commit()
>>> log
[]

The hook is only called for a full commit, not for a savepoint.

>>> t = begin()
>>> t.addBeforeCommitHook(hook, ('A',), dict(kw1='B'))
>>> dummy = t.savepoint()
>>> log
[]
>>> t.commit()
>>> log
["arg 'A' kw1 'B' kw2 'no_kw2'"]
>>> reset_log()

If a transaction is aborted, no hook is called.

>>> from transaction import abort
>>> t = begin()
>>> t.addBeforeCommitHook(hook, ["OOPS!"])
>>> abort()
>>> log
[]
>>> commit()
>>> log
[]

The hook is called before the commit does anything, so even if the commit fails the hook will have been called. To provoke failures in commit, we’ll add failing resource manager to the transaction.

>>> class CommitFailure(Exception):
...     pass
>>> class FailingDataManager:
...     def tpc_begin(self, txn, sub=False):
...         raise CommitFailure('failed')
...     def abort(self, txn):
...         pass

>>> t = begin()
>>> t.join(FailingDataManager())

>>> t.addBeforeCommitHook(hook, ('2',))

>>> from transaction.tests.common import DummyFile
>>> from transaction.tests.common import Monkey
>>> from transaction.tests.common import assertRaisesEx
>>> from transaction import _transaction
>>> buffer = DummyFile()
>>> with Monkey(_transaction, _TB_BUFFER=buffer):
...     err = assertRaisesEx(CommitFailure, t.commit)
>>> log
["arg '2' kw1 'no_kw1' kw2 'no_kw2'"]
>>> reset_log()

Let’s register several hooks.

>>> t = begin()
>>> t.addBeforeCommitHook(hook, ('4',), dict(kw1='4.1'))
>>> t.addBeforeCommitHook(hook, ('5',), dict(kw2='5.2'))

They are returned in the same order by getBeforeCommitHooks.

>>> [(hook.__name__, args, kws)  
...  for hook, args, kws in t.getBeforeCommitHooks()]
[('hook', ('4',), {'kw1': '4.1'}),
('hook', ('5',), {'kw2': '5.2'})]

And commit also calls them in this order.

>>> t.commit()
>>> len(log)
2
>>> log  
["arg '4' kw1 '4.1' kw2 'no_kw2'",
"arg '5' kw1 'no_kw1' kw2 '5.2'"]
>>> reset_log()

While executing, a hook can itself add more hooks, and they will all be called before the real commit starts.

>>> def recurse(txn, arg):
...     log.append('rec' + str(arg))
...     if arg:
...         txn.addBeforeCommitHook(hook, ('-',))
...         txn.addBeforeCommitHook(recurse, (txn, arg-1))

>>> t = begin()
>>> t.addBeforeCommitHook(recurse, (t, 3))
>>> commit()
>>> log  
['rec3',
        "arg '-' kw1 'no_kw1' kw2 'no_kw2'",
'rec2',
        "arg '-' kw1 'no_kw1' kw2 'no_kw2'",
'rec1',
        "arg '-' kw1 'no_kw1' kw2 'no_kw2'",
'rec0']
>>> reset_log()

The addAfterCommitHook() Method

Let’s define a hook to call, and a way to see that it was called.

>>> log = []
>>> def reset_log():
...     del log[:]

>>> def hook(status, arg='no_arg', kw1='no_kw1', kw2='no_kw2'):
...     log.append("%r arg %r kw1 %r kw2 %r" % (status, arg, kw1, kw2))

Now register the hook with a transaction.

>>> from transaction import begin
>>> t = begin()
>>> t.addAfterCommitHook(hook, ('1',))

We can see that the hook is indeed registered.

>>> [(hook.__name__, args, kws)
...  for hook, args, kws in t.getAfterCommitHooks()]
[('hook', ('1',), {})]

When transaction commit is done, the hook is called, with its arguments.

>>> log
[]
>>> t.commit()
>>> log
["True arg '1' kw1 'no_kw1' kw2 'no_kw2'"]
>>> reset_log()

A hook’s registration is consumed whenever the hook is called. Since the hook above was called, it’s no longer registered:

>>> from transaction import commit
>>> len(list(t.getAfterCommitHooks()))
0
>>> commit()
>>> log
[]

The hook is only called after a full commit, not for a savepoint.

>>> t = begin()
>>> t.addAfterCommitHook(hook, ('A',), dict(kw1='B'))
>>> dummy = t.savepoint()
>>> log
[]
>>> t.commit()
>>> log
["True arg 'A' kw1 'B' kw2 'no_kw2'"]
>>> reset_log()

If a transaction is aborted, no hook is called.

>>> from transaction import abort
>>> t = begin()
>>> t.addAfterCommitHook(hook, ["OOPS!"])
>>> abort()
>>> log
[]
>>> commit()
>>> log
[]

The hook is called after the commit is done, so even if the commit fails the hook will have been called. To provoke failures in commit, we’ll add failing resource manager to the transaction.

>>> class CommitFailure(Exception):
...     pass
>>> class FailingDataManager:
...     def tpc_begin(self, txn):
...         raise CommitFailure('failed')
...     def abort(self, txn):
...         pass

>>> t = begin()
>>> t.join(FailingDataManager())

>>> t.addAfterCommitHook(hook, ('2',))
>>> from transaction.tests.common import DummyFile
>>> from transaction.tests.common import Monkey
>>> from transaction.tests.common import assertRaisesEx
>>> from transaction import _transaction
>>> buffer = DummyFile()
>>> with Monkey(_transaction, _TB_BUFFER=buffer):
...     err = assertRaisesEx(CommitFailure, t.commit)
>>> log
["False arg '2' kw1 'no_kw1' kw2 'no_kw2'"]
>>> reset_log()

Let’s register several hooks.

>>> t = begin()
>>> t.addAfterCommitHook(hook, ('4',), dict(kw1='4.1'))
>>> t.addAfterCommitHook(hook, ('5',), dict(kw2='5.2'))

They are returned in the same order by getAfterCommitHooks.

>>> [(hook.__name__, args, kws)     
...  for hook, args, kws in t.getAfterCommitHooks()]
[('hook', ('4',), {'kw1': '4.1'}),
('hook', ('5',), {'kw2': '5.2'})]

And commit also calls them in this order.

>>> t.commit()
>>> len(log)
2
>>> log  
["True arg '4' kw1 '4.1' kw2 'no_kw2'",
"True arg '5' kw1 'no_kw1' kw2 '5.2'"]
>>> reset_log()

While executing, a hook can itself add more hooks, and they will all be called before the real commit starts.

>>> def recurse(status, txn, arg):
...     log.append('rec' + str(arg))
...     if arg:
...         txn.addAfterCommitHook(hook, ('-',))
...         txn.addAfterCommitHook(recurse, (txn, arg-1))

>>> t = begin()
>>> t.addAfterCommitHook(recurse, (t, 3))
>>> commit()
>>> log  
['rec3',
        "True arg '-' kw1 'no_kw1' kw2 'no_kw2'",
'rec2',
        "True arg '-' kw1 'no_kw1' kw2 'no_kw2'",
'rec1',
        "True arg '-' kw1 'no_kw1' kw2 'no_kw2'",
'rec0']
>>> reset_log()

If an after commit hook is raising an exception then it will log a message at error level so that if other hooks are registered they can be executed. We don’t support execution dependencies at this level.

>>> from transaction import TransactionManager
>>> from transaction.tests.test__manager import DataObject
>>> mgr = TransactionManager()
>>> do = DataObject(mgr)

>>> def hookRaise(status, arg='no_arg', kw1='no_kw1', kw2='no_kw2'):
...     raise TypeError("Fake raise")

>>> t = begin()

>>> t.addAfterCommitHook(hook, ('-', 1))
>>> t.addAfterCommitHook(hookRaise, ('-', 2))
>>> t.addAfterCommitHook(hook, ('-', 3))
>>> commit()

>>> log
["True arg '-' kw1 1 kw2 'no_kw2'", "True arg '-' kw1 3 kw2 'no_kw2'"]

>>> reset_log()

Test that the associated transaction manager has been cleaned up when after commit hooks are registered

>>> t = begin()
>>> t._manager is not None
True
>>> t._manager._txn is t
True

>>> t.addAfterCommitHook(hook, ('-', 1))
>>> commit()

>>> log
["True arg '-' kw1 1 kw2 'no_kw2'"]

>>> t._manager is None
True
>>> mgr._txn is None
True

>>> reset_log()