Named Entity Recognizer¶
- class vnlp.named_entity_recognizer.named_entity_recognizer.NamedEntityRecognizer(model='SPUContextNER', evaluate=False)[source]¶
Main API class for Named Entity Recognizer implementations.
Available models: [‘SPUContextNER’, ‘CharNER’]
In order to evaluate, initialize the class with “evaluate = True” argument. This will load the model weights that are not trained on test sets.
- predict(sentence: str, displacy_format: bool = False) List[Tuple[str, str]] [source]¶
High level user API for Named Entity Recognition.
- Parameters:
sentence – Input sentence/text.
displacy_format – When set True, returns the result in spacy.displacy format to allow visualization.
- Returns:
NER result as pairs of (token, entity).
Example:
from vnlp import NamedEntityRecognizer ner = NamedEntityRecognizer() ner.predict("Benim adım Melikşah, 29 yaşındayım, İstanbul'da ikamet ediyorum ve VNGRS AI Takımı'nda çalışıyorum.") [('Benim', 'O'), ('adım', 'O'), ('Melikşah', 'PER'), (',', 'O'), ('29', 'O'), ('yaşındayım', 'O'), (',', 'O'), ("İstanbul'da", 'LOC'), ('ikamet', 'O'), ('ediyorum', 'O'), ('ve', 'O'), ('VNGRS', 'ORG'), ('AI', 'ORG'), ("Takımı'nda", 'ORG'), ('çalışıyorum', 'O'), ('.', 'O')] # Visualization with Spacy: import spacy from vnlp import NamedEntityRecognizer ner = NamedEntityRecognizer() result = ner.predict("İstanbul'dan Foça'ya giderken Zeynep ile Bursa'ya uğradık.", displacy_format = True) spacy.displacy.render(result, style="ent", manual = True)
SentencePiece Unigram Context Named Entity Recognizer¶
- class vnlp.named_entity_recognizer.spu_context_ner.SPUContextNER(evaluate)[source]¶
SentencePiece Unigram Context Named Entity Recognizer class.
This is a context aware Deep GRU based Named Entity Recognizer that uses SentencePiece Unigram tokenizer and pre-trained Word2Vec embeddings.
It achieves 0.9928 Accuracy and 0.9833 F1 score on test sets of “wikiann”, “gungor.ner” and “teghub-TurkishNER-BERT” datasets.
It achieves 0.9766 F1 score for “ORG”, 0.9852 F1 score for “PER” and 0.9742 F1 score for “LOC” entities. (Treating entity of interest as positive, all others as negative class.)
For more details about the training procedure, dataset and evaluation metrics, see ReadMe.
CharNER¶
- class vnlp.named_entity_recognizer.charner.CharNER(evaluate)[source]¶
CharNER Named Entity Recognizer.
This is an implementation of CharNER: Character-Level Named Entity Recognition.
There are slight modifications to the original paper:
This version is trained for Turkish language only.
This version uses simple Mode operation among the character predictions of each token, instead of Viterbi Decoder
It achieves 0.9589 Accuracy and 0.9200 F1_macro_score.
Input data is processed by NLTK.tokenize.WordPunctTokenizer so that each punctuation becomes a new token.
Entity labels are: [‘O’, ‘PER’, ‘LOC’, ‘ORG’]
For more details about the training procedure, dataset and evaluation metrics, see ReadMe.