Writing Python code for both ndarray and symbol

1

I am implementing some fairly complex criterion function in MXNet. I’d like to both support Gluon and the old way of binding executors.
This means I have to write all my code both using mx.nd and mx.sym, apart from that, it is pretty much the same (I am aware there are differences: I can access shape and dtype in mx.nd, but not in mx.sym. I can overwrite array content in mx.nd, have out=XYZ option, but not in mx.sym. But most is the same).

Is there some trick to avoid having to copy & paste?

Sorry if this is obvious.

2

Use Gluon.HybridBlock

3

Hi Eric,
so you say I should abandon my mx.sym code, and instead rather use HybridBlock later on if speed is an issue.

I am happy to do that, but I also need a CustomOp, and it seems what I am doing with mx.sym.Custom does not work with mx.nd.Custom (another topic I just posted).

4

I would start out using a HybridBlock from the beginning. It will use either the NDArray or Symbol APIs based on what mode it is in. You will have to make sure you do not use functions that are not in both APIs, such as implicit broadcasting operations on NDArray. And you may have to implement a CustomOp for some cases.

5

Hello Chris, thanks for your advice.
Gluon is novel to me.

In fact, I am implementing a loss function (albeit a quite complex one). I thought Block are reserved for elements in the middle? Anyway, much to learn. I need to look into HybridBlock. Is there a good tutorial for people who know the “old” MXNet way, to transfer to Gluon?

6

OK, I went through:
http://gluon.mxnet.io/chapter07_distributed-learning/hybridize.html
to understand HybridBlock. I see now how it works.

Maybe just allow me once more question: How can hybridize() work on something like the example in that chapter, if the input dimension (dimension of 1st layer) is not specified?

I’d expect hybridize() to do something like bind executors to all HybridBlock parts of the graph, to make things fast. But for that, I need to know all shapes, I also need to know the input shape, right?

I am asking because the layer (or loss function) I want to write, also has to know the shape of its input. Just wondering if this can somehow be accessed in HybridBlock.hybrid_forward?

7

Hybridize will build your symbol graph but will not bind it. I believe that happens when you call it.

You can also just build a symbol graph by calling hybrid_forward with the input variable and then use it like any other symbol graph.