Lecture

Representing meaning as a dense vector learned from large collections of
raw text is compelling for multiple reasons. Concepts appearing in similar
contexts are assigned similar vectors and therefore are projected close in a
given embedding space. This is critical for enabling pre-training of language
models because linguistic, world and commonsense knowledge is encoded for free
in such vectors without an explicit supervision signal. However, this approach
has limitations: (1) rare words, slang, jargon or neologisms are unlikely
to get a high quality representation due to their relative infrequency in
raw corpora; (2) interpretability is challenging because it is not obvious
which and how facets or properties are weighted in the resulting embedding;
and (3) bias mitigation is left mostly to the quality of pretraining and/or
finetuning corpora, but it would be desirable to have access to a "control
panel" that allows us to tune up or down certain behaviours. In this
talk I will review some works that aim to address some of these challenges,
and will argue that property-based meaning representation has advantages
that should be harnessed in today's fast paced LLM era.

Luis Espinosa Anke is a tech lead in Natural Language Processing at Envelop
Risk Analytics and senior lecturer (~associate professor) at Cardiff
University. His research interests range from computational semantics and
computational lexicography to specialization and steering of large language
models.
In industry, he was previously head of machine learning at AMPLYFI, a
Cardiff-based market intelligence startup, and before that, data scientist
at Nationwide Building Society. He is a Fulbright scholar (2010-2011 cohort)
and recipient of laCaixa scholarship (2011 - 2013). He obtained his PhD
award at Pompeu Fabra University