Term of the Week: Natural Language Processing (NLP)

Why does a technical communicator need to know this?

NLP plays a central role in a wide range of applications today. For example, many spam filters use a technique called Bayesian filtering. This is a statistical approach in which the frequency of certain words are measured against the distribution of similar words in a collection of known spam email messages. Using this approach, we can calculate the probability that a message is spam based on its content. We can use a similar approach to detect the language of a tweet or a website by measuring character distributions.

Another NLP technique, central to every web search, attempts to extract meaning from texts. For example, using NLP techniques, we can recognize search queries, extract the meaning, and provide answers that are contextually relevant to the user. Further refinement of these techniques allow us to summarize the meaning of documents and to develop sentiment analysis, a method of identifying and categorizing opinions in a piece of text to determine whether the attitude of the writer is positive, negative, or neutral. This analysis determines how the launch of a new product is perceived by the market.

Machine translation (MT) is another important application of NLP. The biggest challenge in MT is not translating words, but preserving the meaning of the sentence. This is a complex technological challenge that is at the heart of NLP research today. Neural MT is better at preserving the meaning of a sentence and, consequently, generates better and more fluent translations.

NLP is at the heart of how we currently interact with computer systems and will drive the next evolution in how we interact with them in the future.