Multi-Task Learning學習筆記

紀錄學習MTL過程中讀過的文獻資料

[ 1 ] Multitask learning in TensorFlow with the Head API

#使用Tensorflow建立Multi-head(以Twin-head為例子)def multi_head_cnn_model_fn(features, labels, mode):    # Extract the features
dense = extract_features(features)
# Predictions for each task
predictions_nose = tf.layers.dense(inputs=dense, units=2)
predictions_pose = tf.layers.dense(inputs=dense, units=5)
logits = {'head_nose': predictions_nose, 'head_pose': predictions_pose}
# Optimizer (for both tasks simultaneously)
optimizer = tf.train.AdamOptimizer()
# Two heads
regression_head = tf.contrib.estimator.regression_head(name='head_nose', label_dimension=2)
classification_head = tf.contrib.estimator.multi_class_head(name='head_pose', n_classes=5)
# Multi-head combining two single heads
multi_head = tf.contrib.estimator.multi_head([regression_head, classification_head])
# Return the final model
return multi_head.create_estimator_spec(features, mode, logits, labels, optimizer)

[ 2 ] An Overview of Multi-Task Learning for Deep Learning

入門MTL的基礎觀念

[ 3 ] Multi-Task Learning with Pytorch and FastAI

使用PyTorch實現MTL,本文搭建的Model預測了年齡、性別、種族Notebook

[ 4 ] Regularization — Part 5 Multi-task Learning

簡單帶入MTL觀念

[ 5 ] Deep Multi-Task Learning — 3 Lessons Learned

介紹了在訓練MTL模型時會遇到的問題
Lesson 1 — Combining losses
Lesson 2 — Tuning learning rates
Lesson 3 — Using estimates as features

[ 6 ] Multi-Task Learning with Deep Neural Networks

沒有太多理論,以Tensorflow實作的方式,跟讀者介紹設計MTL的流程

[ 7 ] End-to-End Multi-Task Learning with Attention

[ 8 ] Cross-stitch Networks for Multi-task Learning

[ 9 ] Revisiting Multi-Task Learning in the Deep Learning Era

[ 10] A Survey on Multi-Task Learning

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store