Odtnhj

I have a multi-output problem (multi-label, multi-classification). In brief, the problem regards instrument recognition in polyphonic music, therefore my model needs to be able to predict the instruments (which can be multiple) in a song.

I need to use a CNN for this. The first block of the network model has been omitted since it is composed only by the feature extractor, with all the convolution, pooling and dropout. I am using the high level API of Tensorflow, Estimators.

 # first part omitted, this is last dropout from a fully connected layer
 dropout = tf.layers.dropout(inputs=dense, rate=0.5, training=mode == tf.estimator.ModeKeys.TRAIN)

 print('Shape Dropout', dropout.shape)

 ###########################################################################

 # Logits Layer
 logits = tf.layers.dense(inputs=dropout, units=labels.shape[1])

 print('Shape Logits:', logits.shape)

 predictions = tf.round(tf.sigmoid(logits))

 if mode == tf.estimator.ModeKeys.PREDICT:
 return tf.estimator.EstimatorSpec(mode=mode, predictions=predictions)


 loss = tf.nn.weighted_cross_entropy_with_logits(targets=tf.cast(labels, tf.float32), logits=logits, pos_weight=3, name=None)

 loss = tf.reduce_mean(tf.reduce_sum(loss, axis=1))

 accuracy = tf.reduce_mean(tf.cast(tf.equal(predictions, tf.round(tf.cast(labels, tf.float32))), tf.float32))

 # calculate f1_score, precision and recall for multilabel problem
 f1s = [0, 0, 0]

 labels = tf.cast(labels, tf.float64)
 predictions = tf.cast(predictions, tf.float64)

 for i, axis in enumerate([None, 0]):
 TP = tf.count_nonzero(predictions * labels, axis=axis)
 FP = tf.count_nonzero(predictions * (labels - 1), axis=axis)
 FN = tf.count_nonzero((predictions - 1) * labels, axis=axis)

 precision = tf.truediv(TP, (TP + FP))
 recall = tf.truediv(TP, (TP + FN))
 f1 = 2 * precision * recall / (precision + recall)

 f1s[i] = tf.reduce_mean(f1)

 weights = tf.reduce_sum(labels, axis=0)
 weights /= tf.reduce_sum(weights)

 f1s[2] = tf.reduce_sum(tf.cast(f1, tf.float64) * weights)

 micro, macro, weighted = f1s


 tf.summary.scalar("micro_mLabels", micro)
 tf.summary.scalar("macro_mLabels", macro)
 tf.summary.scalar("weighted_mLabels", weighted)


 tf.summary.scalar("precision_sLabel", tf.metrics.precision(labels=labels, predictions=tf.cast(predictions, tf.int32)))
 tf.summary.scalar("recall_sLabel",tf.metrics.recall(labels=labels, predictions=tf.cast(predictions, tf.int32)))


 if mode == tf.estimator.ModeKeys.TRAIN:
 optimizer = tf.train.AdamOptimizer(learning_rate=0.001)
 train_op = optimizer.minimize(
 loss=loss,
 global_step=tf.train.get_global_step())
 logging_hook = tf.train.LoggingTensorHook("loss": loss, "accuracy": accuracy, every_n_iter=10)
 return tf.estimator.EstimatorSpec(mode=mode, loss=loss, train_op=train_op, training_hooks = [logging_hook])



 eval_metric_ops = 
 "accuracy_sLabel": tf.metrics.accuracy(
 labels=labels,
 predictions=predictions),
 "precision_sLabel": tf.metrics.precision(
 labels=labels,
 predictions=tf.cast(predictions, tf.int32)),
 "recall_sLabel": tf.metrics.recall(
 labels=labels,
 predictions=tf.cast(predictions, tf.int32)),
 

 return tf.estimator.EstimatorSpec(mode=mode, loss=loss, eval_metric_ops=eval_metric_ops)

I recalculate precision and recall internally to check whether was a Tensorflow bug.

Basically, the problem I am having is that the accuracy is really high from the beginning, like in 20 iterations is already at 90% (very weird, should take few epochs), precision and recall differently sometimes are 0 and sometimes not, depending from what hyperparameters I set.

I understand I need to follow a Sigmoid approach for multi-label but I think I am doing things in a wrong way.

What I believe is that the way in which I store the predictions from the logits and in how I calculate the loss is wrong. However, I am not quite sure about the rest of the code, that's why I added the full block.

Really appreciate some help,

Thank you in advance.