SuPar v1.1.3

Release Notes

Highlights

We add implementations of many structured prediction algorithms presented in the form of torch.disrtibutions and semiring notations¹, including

• Tree
  • MatrixTree (Koo et al., 2007; Ma and Hovy, 2017)
  • DependencyCRF (Eisner et al., 2000; Zhang et al., 2020)
  • Dependency2oCRF (McDonald et al., 2006; Zhang et al., 2020)
  • ConstituencyCRF (Stern et al. 2017; Zhang et al., 2020b)
• Linear Chain:
  • LinearChainCRF (Lafferty et al., 2001)

Take LinearChainCRF as an example:

>>> from supar import LinearChainCRF
>>> batch_size, seq_len, n_tags = 2, 5, 4
>>> lens = torch.tensor([3, 4])
>>> value = torch.randint(n_tags, (batch_size, seq_len))
>>> s1 = LinearChainCRF(torch.randn(batch_size, seq_len, n_tags),
                        torch.randn(n_tags+1, n_tags+1),
                        lens)
>>> s2 = LinearChainCRF(torch.randn(batch_size, seq_len, n_tags),
                        torch.randn(n_tags+1, n_tags+1),
                        lens)
>>> s1.max
tensor([4.4120, 8.9672], grad_fn=<MaxBackward0>)
>>> s1.argmax
tensor([[2, 0, 3, 0, 0],
        [3, 3, 3, 2, 0]])
>>> s1.log_partition
tensor([ 6.3486, 10.9106], grad_fn=<LogsumexpBackward>)
>>> s1.log_prob(value)
tensor([ -8.1515, -10.5572], grad_fn=<SubBackward0>)
>>> s1.entropy
tensor([3.4150, 3.6549], grad_fn=<SelectBackward>)
>>> s1.kl(s2)
tensor([4.0333, 4.3807], grad_fn=<SelectBackward>)

Bug fixes

• Fix bug of model saving (#82)
• Fix issue of stride setting for small BERT (#86)
• Fix preprocessing crashes for some UD treebanks (#85)

1. The implementations of structured distributions and semirings are heavily borrowed from torchstruct with some tailoring. For more details, see their tutorial paper and Goodman's paper. ↩