GloVe: Word Vectors From a Global Co-Occurrence Matrix, Not Only From Local Windows
Pennington, Socher, and Manning factorize a log co-occurrence matrix with a weighted least-squares objective. When count-based vectors still beat a hastily trained Word2Vec, and when contextual models replaced both.