Named Entity Recognizer

class vnlp.named_entity_recognizer.named_entity_recognizer.NamedEntityRecognizer(model='SPUContextNER', evaluate=False)

Main API class for Named Entity Recognizer implementations.

Available models: [‘SPUContextNER’, ‘CharNER’]

In order to evaluate, initialize the class with “evaluate = True” argument. This will load the model weights that are not trained on test sets.

predict(sentence: str, displacy_format: bool = False) → List[Tuple[str, str]]

High level user API for Named Entity Recognition.

Parameters:

• sentence – Input sentence/text.

• displacy_format – When set True, returns the result in spacy.displacy format to allow visualization.

Returns:

NER result as pairs of (token, entity).

Example:

from vnlp import NamedEntityRecognizer
ner = NamedEntityRecognizer()
ner.predict("Benim adım Melikşah, 29 yaşındayım, İstanbul'da ikamet ediyorum ve VNGRS AI Takımı'nda çalışıyorum.")

[('Benim', 'O'),
('adım', 'O'),
('Melikşah', 'PER'),
(',', 'O'),
('29', 'O'),
('yaşındayım', 'O'),
(',', 'O'),
("İstanbul'da", 'LOC'),
('ikamet', 'O'),
('ediyorum', 'O'),
('ve', 'O'),
('VNGRS', 'ORG'),
('AI', 'ORG'),
("Takımı'nda", 'ORG'),
('çalışıyorum', 'O'),
('.', 'O')]

# Visualization with Spacy:
import spacy
from vnlp import NamedEntityRecognizer
ner = NamedEntityRecognizer()
result = ner.predict("İstanbul'dan Foça'ya giderken Zeynep ile Bursa'ya uğradık.", displacy_format = True)
spacy.displacy.render(result, style="ent", manual = True)

SentencePiece Unigram Context Named Entity Recognizer

class vnlp.named_entity_recognizer.spu_context_ner.SPUContextNER(evaluate)

SentencePiece Unigram Context Named Entity Recognizer class.

• This is a context aware Deep GRU based Named Entity Recognizer that uses SentencePiece Unigram tokenizer and pre-trained Word2Vec embeddings.

• It achieves 0.9928 Accuracy and 0.9833 F1 score on test sets of “wikiann”, “gungor.ner” and “teghub-TurkishNER-BERT” datasets.

• It achieves 0.9766 F1 score for “ORG”, 0.9852 F1 score for “PER” and 0.9742 F1 score for “LOC” entities. (Treating entity of interest as positive, all others as negative class.)

• For more details about the training procedure, dataset and evaluation metrics, see ReadMe.

predict(sentence: str, displacy_format: bool = False) → List[Tuple[int, str, int, str]]

Parameters:

• sentence – Input sentence/text.

• displacy_format – When set True, returns the result in spacy.displacy format to allow visualization.

Returns:

NER result as pairs of (token, entity).

CharNER

class vnlp.named_entity_recognizer.charner.CharNER(evaluate)

CharNER Named Entity Recognizer.

• This is an implementation of CharNER: Character-Level Named Entity Recognition.

• There are slight modifications to the original paper:

• This version is trained for Turkish language only.

• This version uses simple Mode operation among the character predictions of each token, instead of Viterbi Decoder

• It achieves 0.9589 Accuracy and 0.9200 F1_macro_score.

• Input data is processed by NLTK.tokenize.WordPunctTokenizer so that each punctuation becomes a new token.

• Entity labels are: [‘O’, ‘PER’, ‘LOC’, ‘ORG’]

• For more details about the training procedure, dataset and evaluation metrics, see ReadMe.

predict(text: str, displacy_format: bool = False) → List[Tuple[str, str]]

Parameters:

• text – Input text.

• displacy_format – When set True, returns the result in spacy.displacy format to allow visualization.

Returns:

NER result as pairs of (token, entity).