General purpose Attention Layer?

1

Hi, All,
Would like to get a general purpose attention layer implementation in MxNet if possible.
Any link/guide is appreciated.

EDIT:
I know sockeye has quite some models but it requires a whole package while I would prefer a very simple layer if possible.

Thanks,

2

There is a GluonNLP toolkit has been introduced recently. It is a separate package that works using MXNet. It has an AttentionCell as a part of it.

Here is the Github - https://github.com/dmlc/gluon-nlp

Please notice, that it works only with master version of mxnet, so you will have to update to pip install --pre --upgrade mxnet before you can start using NLP Toolkit.

3

Appreciate this. I was informed once of this existence but didn’t pay attention. It’s really nice to have this in place though it could use more documentation and example :slight_smile: