How to use a symbol with its "name" is known

Discussion
1

I want to implement l1 regularization with symbol APIs, and I am struggled with the collecting of all the parameters.
I can use out.list_arguments() to get all the trainable arguments, but list_arguments() only list their name, rather than python symbols. Here comes my question.

It is quite easily create a new symbol by

import mxnet as mx
foo=mx.sym.var('bar')

Here, we create a python variable names foo, but its name in mxnet is 'bar'
I want to know can we use 'bar' rather than foo
Seems difficult since we have:

f0=mx.sym.var('bar',shape=(3,3))
f1=mx.sym.var('bar',shape=(2,2))
>>> f0.infer_shape()
([(3, 3)], [(3, 3)], [])

Further more, I donā€™t know if it is polite to ask another question in this thread, but I really want to know if there exists a better way to perform mx.*.dot between mx.nd.array and mx.sym.var
If we already knows some ā€˜ground truthā€™, we may come up with a constant matrix (mx.nd.array), but the data is unkonwn (mx.sym.var). If we want to preform a dot op, what ever we use(mx.nd.dot/mx.sym.dot), mxnet will raise an exception.

I finally came up with two solutions, one of them is

STx=mx.sym.stop_gradient(mx.sym.var('ST',shape=ST.shape,init=mx.init.Constant(ST.asnumpy().tolist())))

and another is

TTx=mx.sym.var('TT',shape=TT.shape)#model should provide the value of STx,YTx,TTx

I donā€™t know if there is a better way to achieve the goal.

2

Regarding writing your own L1 regularizationā€¦ I looked around and it seems like the way to do it is actually implementing your own Optimizer which will have L1 regularization logic. You can do it using Python by basically following how it is done in MXNet code.

Here is the excerpt from the docs:

For optimizer , create a subclass of Optimizer and implement two methods create_state and update . Also add @mx.optimizer.Optimizer.register before this class. See optimizer.py for examples.

1 Like
3

As for the second question, Symbol and NDArray cannot really be used together in an operation. Symbols do point to NDArray under the hood, but the operations can be done either when both operands are Symbols or both of them are NDArrays.