Multi-Task Learning學習筆記
紀錄學習MTL過程中讀過的文獻資料
[ 1 ] Multitask learning in TensorFlow with the Head API
#使用Tensorflow建立Multi-head(以Twin-head為例子)def multi_head_cnn_model_fn(features, labels, mode): # Extract the features
dense = extract_features(features) # Predictions for each task
predictions_nose = tf.layers.dense(inputs=dense, units=2)
predictions_pose = tf.layers.dense(inputs=dense, units=5)
logits = {'head_nose': predictions_nose, 'head_pose': predictions_pose} # Optimizer (for both tasks simultaneously)
optimizer = tf.train.AdamOptimizer() # Two heads
regression_head = tf.contrib.estimator.regression_head(name='head_nose', label_dimension=2)
classification_head = tf.contrib.estimator.multi_class_head(name='head_pose', n_classes=5) # Multi-head combining two single heads
multi_head = tf.contrib.estimator.multi_head([regression_head, classification_head]) # Return the final model
return multi_head.create_estimator_spec(features, mode, logits, labels, optimizer)
[ 2 ] An Overview of Multi-Task Learning for Deep Learning
入門MTL的基礎觀念[ 3 ] Multi-Task Learning with Pytorch and FastAI
使用PyTorch實現MTL，本文搭建的Model預測了年齡、性別、種族Notebook
[ 4 ] Regularization — Part 5 Multi-task Learning
簡單帶入MTL觀念[ 5 ] Deep Multi-Task Learning — 3 Lessons Learned
介紹了在訓練MTL模型時會遇到的問題
Lesson 1 — Combining losses
Lesson 2 — Tuning learning rates
Lesson 3 — Using estimates as features[ 6 ] Multi-Task Learning with Deep Neural Networks
沒有太多理論，以Tensorflow實作的方式，跟讀者介紹設計MTL的流程[ 7 ] End-to-End Multi-Task Learning with Attention
[ 8 ] Cross-stitch Networks for Multi-task Learning
[ 9 ] Revisiting Multi-Task Learning in the Deep Learning Era
[ 10] A Survey on Multi-Task Learning
[11] Branched Multi-Task Networks: Deciding What Layers To Share
[12] Learning to Branch for Multi-Task Learning
[13]AdaShare: Learning What To Share For Efficient Deep Multi-Task Learning
[14]Which Tasks Should Be Learned Together in Multi-task Learning?
[15]Multi-task Learning and Beyond: 过去，现在与未来
作者為End-to-End Multi-Task Learning with Attention的論文作者