This talk will consist of two interweaving parts on the border of machine learning and machine translation. In the theoretical part I will focus on large-scale multiclass classification, online learning, label-dependent features and a choice of loss function. In the practical part I will report on my ongoing experiments with Vowpal Wabbit and deep-syntactic MT system TectoMT.