Hi everyone,
I am new to mxnet. I am working with a dataset for which I know that the model and the data allow a good classification.
I observe the following phenomenon: I am training a simple feed forward network using mx.model.FeedForward.create reaching an accuracy in the 80s. If I apply the returned model to the traning data and check the confusion matrix as well as the accuracy, it’s just about 3%.
I tried the mxnet examples (on mxnet.apache.org) to assure that my installation in R works correclty.
I’ve searched the web a lot and tried all different options but can’t find what the issue is. Hopefully somebody can help.
Thanks in advance for your replies!
Björn
I am pasting the code that I am using here. Is there any way to attach some files? I’ve got an .r-file and an .RData so that people could try to redo the example on their own:
require(mxnet)
# optional. Just for visualization of confusion matrix
require(caret)
nn.layer.input <- mx.symbol.Variable('data')
nn.layer.fc_3 <- mx.symbol.FullyConnected(data = nn.layer.input, num_hidden = 4)
NN_model <- mx.symbol.SoftmaxOutput(data = nn.layer.fc_3)
mx.set.seed(0)
devices <- mx.cpu()
model <- mx.model.FeedForward.create(NN_model,
X = data.x.train,
y = data.y.train,
ctx = devices,
array.batch.size = 32,
optimizer = "adam",
begin.round = 10,
num.round = 50,
learning.rate = 1e-3,
eval.metric = mx.metric.accuracy)
prob.y.predict <- predict(model, data.x.train, array.batch.size = 1)
y.predict <- max.col(t(prob.y.predict))
table(y.predict, data.y.train) #Here you can already see that there is too much of confusion for being in the 80s
# optional if caret is available
confusionMatrix(factor(y.predict, 1:4), factor(data.y.train,1:4))