Have you ever wondered how search engines provide relevant results when you type in a query? Or how virtual assistants like Siri or Alexa can understand and respond to your requests? One of the key technologies behind these applications is named entity recognition, or NER.
NER is a powerful and versatile tool that is used in many different fields and industries to find and classify named entities in text.
In this blog, we will explore the basics of named entity recognition and learn to use named entity recognition models with Python.
What Is Named Entity Recognition?
To understand Named Entity Recognition, we first explore what a named entity is.
Named Entity
A named entity is a specific word or phrase that refers to a particular person, place, organization, money, time or other real-world values.
Named entities are important in Natural Language Processing (NLP) because they provide valuable information and context about the text.
For example, a named entity like "Tom Cruise" can tell us that the text is referring to a specific person rather than just the general concept of "actors."
Named Entity Recognition
Named Entity Recognition (NER) is an NLP technique to find and classify entities from textual data into predefined categories called named entities. This can include entities like an organization, an individual's name, location, a product, etc.
Types of Named Entities
There are many different types of named entities. Each type gives a domain or highlights some information in the text. Some common types of named entities include:
- People—name of an individual, e.g: Tom Cruise or Lionel Messi.
- Organizations—name of a company, organization, or an institution, such as Microsoft Corporation or Stanford University.
- Locations—name of places, such as Istanbul, Turkey or Mount Everest.
- Products—name of products, such as Macbook or Coca-Cola.
- Events—name of events, such as World War I or FIFA World Cup.
Where Is Named Entity Recognition Used?
NER is used in many different applications in NLP. Some of the common uses of NER include:
- Information extraction: NER is used to automatically extract specific named entities from text and store them in a structured format, like a database. This information is then used for purposes, such as generating reports or building knowledge graphs.
- Search engine performance optimization: A named entity recognition (NER) model can be used to tag articles with relevant entities (e.g. people, organizations, locations). These tags can then be stored and used to quickly and efficiently match search queries with relevant articles. This can save computational resources and improve search speed.
- Text summarization: We can use NER to find important named entities in a text or a document and use them to give a summary of the text with contextual information highlighted.
Overall, NER is a valuable tool in natural language processing that has many applications in helping systems understand and extract information from text.
NER Using Python with Pre-Trained Models
Performing named entity recognition with a pre-trained model using Python typically involves the following steps:
- Choosing the NER Library
spaCy, nltk, and flair are all open-source libraries for natural language processing (NLP) in Python. They all include various tools and features for working with text, such as tokenization, part-of-speech tagging, and named entity recognition (NER).
- spaCy is a library for NLP that is designed to be fast and efficient, with a focus on production-grade usage. It includes pre-trained models for many common NLP tasks, including NER and offers a simple and intuitive API for working with text.
- nltk is a more general-purpose library for NLP, widely used by researchers, with a wider range of tools and features. It includes a module for NER, but the implementation is not as efficient or well-optimized as the one in spaCy.
- flair is a library for NLP that is focused on state-of-the-art performance and advanced techniques, such as transfer learning. It includes a pre-trained model for NER that is based on a neural network and offers good performance on a variety of tasks.
Which One Is Better?
In terms of which is better, it depends on your specific requirements and preferences.
- For Fast and Efficient NER:
spaCy is a good option if you need high-velocity NER implementation that is easy to use and well-suited for production environments. - For a Greater Range of Tools and Features:
nltk is a good option if you need a wider range of NLP tools and features. It is highly efficient in sentence tokenization but may not be as efficient or well-optimized as spaCy in work tokenization and parts-of-speech tagging. - For State-of-the-Art Performance and Techniques:
flair is a good option if you need state-of-the-art performance and advanced techniques, but may be more complex to use and require more resources.
Ultimately, the best choice will depend on your specific needs and goals. For this blog, we will use spaCy for its easy implementation. Some of the key features of spaCy include:
- Tokenization
- Part-of-speech tagging
- Named entity recognition
- Dependency parsing
- Sentence boundary detection
- Similarity analysis
- Installing the Required Packages
For NER using Python, you'll need to install a language-based Python package among the choices discussed previously. Since we are using spaCy here, we can install the package using the pip command, for example:
The package already contains some pre-trained NER models.
After the package is installed, import the library into your Python code and use them to perform NER.
Note that these packages may also require additional data and models to be downloaded in order to perform NER. For example, spaCy includes a number of pre-trained models for different languages and tasks, and you will need to download the specific model for your use. You can do this using the spaCy download command, for example:
This will download the small English model for use with spaCy. You can then load this model in your Python code and use it to perform NER.
- Loading a Pre-Trained Model
Once you have installed the necessary Python packages, you can load a pre-trained model for named entity recognition (NER) and specify the named entity categories that you want to recognize.
For example, using the spaCy package, you could load the English model and specify the categories "PERSON," "ORG," "GPE," and "PRODUCT" like this:
In this code, the spaCy package is imported, and the English model is loaded using the spacy.load() method. Then, a list of named entity categories is defined, specifying the categories of named entities that we want to recognize.
- Tokenizing the Text
Once the pre-trained model is loaded and the named entity categories are specified, you can tokenize the text that you want to perform NER on, and then use the model to identify and classify named entities in the text.
For example, using the spaCy package, you could tokenize the text and then use the model to identify and classify named entities like this:
- Identifying and Classifying Named Entities
After tokenization, you can use the doc.ents property of the Doc object to extract the named entities from the tokenized text, and then iterate through the named entities and classify them based on the specified categories.
The code checks whether the entity belongs to one of the specified named entity categories using the ent.label_ property, and if so, it appends the text and category of the entity to a list of entities. This list will be used later to print the named entities and their categories.
Next, the named entities are extracted from the tokenized text using the doc.ents property, and only the entities that belong to one of the four specified categories ("PERSON," "ORG," "GPE," and "PRODUCT") are extracted and stored in a list.
- Displaying the Named Entities
Once you have used the pre-trained model to identify and classify the named entities in the text, you can print the named entities and their categories to see the results of the NER.
The code above gives the following results:
- Visualizing the Results
To visualize the named entity recognition (NER) results with spaCy, you can use the spacy.displacy.render() function to generate an interactive visualization of the entities in a given text.
This function takes a Doc object, which represents a document that has been processed by a spaCy NER model, and it returns a visualization of the entities in the document.
Here is an example of how to visualize the NER results with spaCy:
The code above, on running in Jupyter Notebook, shows the output as below:
Limitations of Using Pre-Trained Models
Using pre-trained models for NER with Python has some limitations, including:
- Not being suitable for all tasks/domains
- Lack of customization/adaptability
- Being resource-intensive to use
Explore Wisecube's Biomedical Tools with State-of-the-Art NER Models
By using Python and pre-trained models, developers and researchers can easily perform NER on a range of texts and languages.
NER is especially a valuable tool for biomedical research. It enables researchers to extract and analyze information from text data, leading to new insights and knowledge.
Wisecube offers a user-friendly interface that allows users to easily access and analyze millions of biomedical publications, reports, and clinical trials using advanced natural language processing techniques like NER. This helps to extract valuable insights and form a rich knowledge base of biomedical information.Schedule a call with us to explore and learn more.