Auxiliary Learning

Self-Supervised Generalisation with Meta Auxiliary Learning 중..

 

Learning with auxiliary tasks can improve the ability of a primary task to generalise.

보조 task와 함께 학습하는 것은 primary task의 일반화 능력을 향상시킬 수 있음.

 

Intro

Auxilairy learning is a method to improve the ability of a primary task to generalise to unseen data, by training on additional auxiliary tasks alongside this primary task.

auxiliary learning은 primary task와 함께 추가적인 auxiliary tasks를 학습함으로서, 한 번도 보지 못한 데이터에 대해서 primary task의 일반화 능력을 향상시키는 방법이다.

The shairing of features across tasks results in additional relevant features being available, which otherwise would not have been learned from training only on the primary task.

tasks 간의 feature 공유로 인해 추가적인 관련 feature를 이용 가능해지며, 이러한 feature들은 primary task만으로 학습할 때는 학습되지 않았을 것이다.

The boader support of these features, across new interpretations of input data, then allows for better generalisation of the primary task.

이러한 feature들의 더 넓은 support는 input data의 새로운 해석을 통해 주요 작업의 더 나은 일반화를 가능하게 한다.

Auxiliary learning is similar to multi-task learning, except that only the performance of the primary task is of importance, and the auxiliary tasks are included purely to assist the primary task.

auxiliary learning은 multi-task learning과 유사하지만, primary task의 성능만 중요하며, auxiliary tasks는 순수하게 주요 작업을 돕기 위해 포함된다.

 

In supervised auxiliary learning, auxiliary tasks can be manually chosen to complement the primary task.

supervised auxiliary learning에서, auxiliary tasks는 primary task를 보완하도록 수동적으로 선택될 수 있다.

However, this requiers both domain knowledge to choose the auxiliary tasks, and labelled data to train the auxiliary tasks.

하지만 적절한 auxiliary tasks를 선택하기 위해 해당 분야의 지식과 auxiliary task를 훈련시키기 위한 label이 필요하다.

 

Related work

Whilst in multi-task learning the goal is high test accuracy across all tasks, auxiliary learning differs in that high test accuracy is only required for a single primary task, and the role of the auxiliary tasks is to assist in generalisation of this primary task.

multi-task에서는 모든 task에 대해서 높은 test accuracy를 목표로 하지만, auxiliary learning은 높은 test accuracy가 primary task에만 필요하며, auxiliary task의 역할은 primary task의 일반화를 돕는 것이다.

Applying reated learning tasks is one straightforward approach to assist primary tasks.

관련 learning tasks를 적용하는 것은 primary task를 assist하는 한 가지 간단한 approach이다.

[33] applied auxiliary supervision with phoneme recognition at intermediate low-level representations to improve the performance of conversational speech recognition.

[33]은 대화식 음성 인식의 성능을 향상시키기 위해 중간 low-level representation에서 음소(phoneme) 인식에 auxiliary supervision을 적용하였다.

[21] chose auxiliary tasks which can be obtained with low effort, such as global descriptions of a scene, to boost the performance for single scene depth estimation and semantic segmentation.

[21]은 scene의 global한 설명과 같이 쉽게 얻을 수 있는 auxiliary tasks를 선택하여 single scene depth estimation 및 semantic segmentation의 성능을 향상시켰다.

By carefully choosing a pair of learning tasks, we may also perform auxiliary learning without ground truth labels, in an unsupervised manner.

a pair of learning tasks를 신중하게 선택하면, unsupervied 방식으로 auxiliary learning을 할 수 있다.

[9] proposed to use cosine similarity as an adaptive task weighting to determine when a defined auxiliary task is useful.

[9]는 정의된 auxiliary task가 유용한 시점을 결정하기 위해 cosine similarity를 적응적 task 가중치로 사용하는 것을 제안했다?

 

 

[21] Lukas Liebel and Marco Körner. Auxiliary tasks in multi-task learning. arXiv preprint arXiv:1805.06334, 2018.

[9] Yunshu Du, Wojciech M Czarnecki, Siddhant M Jayakumar, Razvan Pascanu, and Balaji Lakshminarayanan. Adapting auxiliary losses using gradient similarity. arXiv preprint arXiv:1812.02224, 2018.