spaCy WordNet

spaCy Wordnet is a simple custom component for using WordNet, MultiWordnet and WordNet domains with spaCy.

The component combines the NLTK wordnet interface with WordNet domains to allow users to:

• Get all synsets for a processed token. For example, getting all the synsets (word senses) of the word bank.
• Get and filter synsets by domain. For example, getting synonyms of the verb withdraw in the financial domain.

Getting started

The spaCy WordNet component can be easily integrated into spaCy pipelines. You just need the following:

Prerequisites

• Python 3.X
• spaCy

You also need to install the following NLTK wordnet data:

bash python -m nltk.downloader wordnet python -m nltk.downloader omw

Install

bash pip install spacy-wordnet

Supported languages

Almost all Open Multi Wordnet languages are supported.

Usage

Once you choose the desired language (from the list of supported ones above), you will need to manually download a spaCy model for it. Check the list of available models for each language at SpaCy 2.x or SpaCy 3.x.

English example

Download example model: bash python -m spacy download en_core_web_sm

Run: ````python

import spacy

from spacywordnet.wordnetannotator import WordnetAnnotator

Load an spacy model

nlp = spacy.load('encoreweb_sm')

Spacy 3.x

nlp.addpipe("spacywordnet", after='tagger')

Spacy 2.x

nlp.addpipe(WordnetAnnotator(nlp, name="spacywordnet"), after='tagger')

token = nlp('prices')[0]

wordnet object link spacy token with nltk wordnet interface by giving acces to

synsets and lemmas

token..wordnet.synsets() token..wordnet.lemmas()

And automatically tags with wordnet domains

token..wordnet.wordnetdomains() ````

spaCy WordNet lets you find synonyms by domain of interest for example economy ````python economydomains = ['finance', 'banking'] enrichedsentence = [] sentence = nlp('I want to withdraw 5,000 euros')

For each token in the sentence

for token in sentence: # We get those synsets within the desired domains synsets = token..wordnet.wordnetsynsetsfordomain(economydomains) if not synsets: enrichedsentence.append(token.text) else: lemmasforsynset = [lemma for s in synsets for lemma in s.lemmanames()] # If we found a synset in the economy domains # we get the variants and add them to the enriched sentence enrichedsentence.append('({})'.format('|'.join(set(lemmasforsynset))))

Let's see our enriched sentence

print(' '.join(enriched_sentence))

>> I (need|want|require) to (draw|withdraw|drawoff|takeout) 5,000 euros

````

Portuguese example

Download example model: bash python -m spacy download pt_core_news_sm

Run: ```python import spacy

from spacywordnet.wordnetannotator import WordnetAnnotator

Load an spacy model

nlp = spacy.load('ptcorenews_sm')

Spacy 3.x

nlp.addpipe("spacywordnet", after='tagger', config={'lang': nlp.lang})

Spacy 2.x

nlp.add_pipe(WordnetAnnotator(nlp.lang), after='tagger')

text = "Eu quero retirar 5.000 euros" economydomains = ['finance', 'banking'] enrichedsentence = [] sentence = nlp(text)

For each token in the sentence

for token in sentence: # We get those synsets within the desired domains synsets = token..wordnet.wordnetsynsetsfordomain(economydomains) if not synsets: enrichedsentence.append(token.text) else: lemmasforsynset = [lemma for s in synsets for lemma in s.lemmanames('por')] # If we found a synset in the economy domains # we get the variants and add them to the enriched sentence enrichedsentence.append('({})'.format('|'.join(set(lemmasforsynset))))

Let's see our enriched sentence

print(' '.join(enriched_sentence))

>> Eu (querer|desejar|esperar) retirar 5.000 euros

```