Skip to yearly menu bar Skip to main content


Poster

Enabling Few-Shot Learning with PID Control: A Layer Adaptive Optimizer

Le Yu · Xinde Li · Pengfei Zhang · zhentong zhang · Fir Dunkin


Abstract:

Model-Agnostic Meta-Learning (MAML) and itsvariants have shown remarkable performance inscenarios characterized by a scarcity of labeleddata during the training phase of machine learningmodels. Despite these successes, MAMLbasedapproaches encounter significant challengeswhen there is a substantial discrepancy in thedistribution of training and testing tasks, resultingin inefficient learning and limited generalizationacross domains. Inspired by classicalproportional-integral-derivative (PID) control theory,this study introduces a Layer-Adaptive PID(LA-PID) Optimizer, a MAML-based optimizerthat employs efficient parameter optimizationmethods to dynamically adjust task-specific PIDcontrol gains at each layer of the network, conductinga first-principles analysis of optimal convergenceconditions. A series of experimentsconducted on four standard benchmark datasetsdemonstrate the efficacy of the LA-PID optimizer,indicating that LA-PID achieves state-ofthe-art performance in few-shot classification andcross-domain tasks, accomplishing these objectiveswith fewer training steps. Code is availableon https://github.com/yuguopin/LA-PID.

Live content is unavailable. Log in and register to view live content