Extracting Exact Dates from Natural Language Text

Using machine learning and regex patters to identify and extract date information in Spark NLP

Extracting date information from text is a common task in Natural Language Processing (NLP) that can be achieved using a variety of techniques:

DateMatcher and MultiDateMatcher are rule-based annotators in the Spark NLP library that are used to extract date expressions from text using pattern matching. DateMatcher can only extract one date per input document while MultiDateMatcher can extract multiple dates; other than that their performances are the same.

DateMatcher can identify a wide range of date formats, including both absolute and relative dates. DateMatcher uses rules to identify and extract date expressions from text. The component is highly customizable, allowing users to specify their own rules to match specific date formats.

Some of the features of Spark NLP DateMatcher include:

Overall, DateMatcher is a powerful and flexible tool for extracting date expressions from text, and can be customized to match specific formats and languages. It is an important component in many NLP applications that require date information, such as news analysis, social media monitoring, and financial forecasting.

In this post, you will learn how to use Spark NLP to perform date extraction from text.

Let us start with a short Spark NLP introduction and then discuss the details of date extraction with some solid results.

Since its first release in July 2017, Spark NLP has grown in a full NLP tool, providing:

Spark NLP comes with 14,500+ pretrained pipelines and models in more than 250+ languages. It supports most of the NLP tasks and provides modules that can be used seamlessly in a cluster.

Spark NLP processes the data using Pipelines, structure that contains all the steps to be run on the input data:

An annotator in Spark NLP is a component that performs a specific NLP task on a text document and adds annotations to it. An annotator takes an input text document and produces an output document with additional metadata, which can be used for further processing or analysis. For example, a named entity recognizer annotator might identify and tag entities such as people, organizations, and locations in a text document, while a sentiment analysis annotator might classify the sentiment of the text as positive, negative, or neutral.

To install Spark NLP in Python, simply use your favorite package manager (conda, pip, etc.). For example:

Then, simply import the library and start a Spark session:

DateMatcher and MultiDateMatcher annotators extract the following date information automatically from the text:

DateMatcher and MultiDateMatcher annotators expect DOCUMENT as input, and then will provide DATEas output.

Please check the details of the pipeline below, where we define a short pipeline and then define 5 texts for text classification:

After that, we get predictions by transforming the model.

The above dataframe shows that DateMatcher extracted one date and MultiDateMatcher extracted all the date information from the text.

Also, date formats for the annotators are different in the pipeline, defined by the setOutputFormat parameter.

DateMatcher and MultiDateMatcher annotators can also return relative dates. To accomplish this, a reference (or anchor) date must be defined. Reference date parameters can be set by setAnchorDateDay(), setAnchorDateMonth(), setAnchorDateYear().

If an anchor date parameter is not set, the current day or current month or current year will be set as the default value.

Check the below pipeline with two separate MultiDateMatcher annotators:

After that, we get predictions by transforming the model for the new parameters:

Different anchor (reference) days produced different results for the 3rd and 5th texts, where reference dates are used.

Date matching annotators can be used with a total of 204 languages. The default value is "en"- English.

The example above shows date extraction in German.

The first row contains an actual date while the second one has a relative date (morgen means tomorrow in English). They are formatted in the desired output format.

For additional information, please consult the following references.

Date information extraction is a crucial task in NLP that plays a vital role in a wide range of text-based applications. By accurately identifying and normalizing dates mentioned in text data, NLP models can help facilitate temporal analysis, event extraction, sentiment analysis, summarization, and data integration. The ability to extract date information from text data helps to unlock valuable insights and drive more advanced text-based applications in a variety of fields.

DateMatcher and MultiDateMatcher are powerful and accurate language detection tools in Spark NLP, and are valuable assets for any NLP application that involves multilingual date value extraction.

Related posts:

Making money online without investment is a topic that has been around for years. With the internet becoming more accessible worldwide, it’s easier than ever to find ways to make money online. In…

Here on the Story Maps Developers’ Corner, we share a lot of tips and techniques about customizing story maps for people who know a little code as well as expert web developers. If you are closely…

Education is characterized as an important human value, a must-have of a society, basis of a well-qualified life and flourishing future conditions. Education is an essential tool for combination of…