Can someone check if the expression should be
XW + b and Hw + b
instead of
WX + b and WH + b
I think that’s just a choice of how you define X. I.e if it’s n\times d, then you will want to right multiply it by a d\times 1 weight vector to get an n\times 1 output, like you mention. Or you could take X to be transpose and d\times n, in which case you’d have the latter setup. They’re equivalent as long as you specify how X is defined.