Li, J., & Armstrong, B. C. (2023). Probing the Representational
Structure of Regular Polysemy in a Contextual Word Embedding Model via
Sense Analogy Questions. Proceedings of the Annual Meeting of the
Cognitive Science Society.
Download:
Author's self-archived version (.pdf) (7 pages)
Official version hosted by the Cognitive Science Society [external link]
Abstract
Regular polysemes are sets of ambiguous words that all share the same relationship between their meanings, such as CHICKEN and LOBSTER both referring to an animal or its meat. To probe how a context embedding model, here exemplified by BERT, represents regular polysemy, we analyzed whether its embeddings support answering sense analogy questions similar to “is the mapping between CHICKEN (as an animal) and CHICKEN (as a meat) the same as that which maps between LOBSTER (as an animal) to LOBSTER (as a meat)?” We found that (1) the model was sensitive to the shared structure within a regularity type; (2) the shared structure varies across regularity types, potentially reflective of a “regularity continuum;” (3) some high-order latent structure may be shared across regularity types, suggestive of a similar latent structure across types; and (4) there is equivocal evidence that the aforementioned effects are explained by meaning overlap.
Keywords: regular polysemy; semantic ambiguity; word analogy; contextual word embeddings; lexical semantics; BERT model
Copyright Notice (borrowed from David Plaut): The documents distributed here have been provided as a means to ensure timely dissemination of scholarly and technical work on a noncommercial basis. Copyright and all rights therein are maintained by the authors or by other copyright holders, notwithstanding that they have offered their works here electronically. It is understood that all persons copying this information will adhere to the terms and constraints invoked by each author's copyright. These works may not be reposted without the explicit permission of the copyright holder.