The NLP Landscape from 1960s to 2020

NLP

What is NLP?

NLP is a sub-field of linguistics, Computer Science & A.I concerned with interactions between computers & human language. In particular how to program computers to process & analyze large amounts of Natural Language data.

Goal : Making machines understand the Natural Language

Some Real World Applications :

1. Contextual Advertisement
2. Email Clients — Spam Filtering, Smart Replies
3. Social Media — Removing Adult Content, Opinion mining
4. Search Engines
5. Chat Bots

Common NLP Tasks:

1. Text/Document Classifications
2. Sentiment Analysis
3. Information Retrieval
4. Parts of Speech tagging
5. Language Detection & Machine Translation
6. Conversational Agents
7. Knowledge Graphs & Q.A sys
8. Text Summarization
9. Topic Modelling
10. Text Generation
11. Spell Checking & Grammar correction
12. Text Parsing
13. Speech to text & Text to Speech

Approaches to NLP:

1. Heuristic Methods
2. M.L based models
3. D.L based models

Heuristic Methods:

A heuristic, or a heuristic technique, is any approach to problem-solving that uses a practical method or various shortcuts in order to produce solutions that may not be optimal but are sufficient given a limited timeframe or deadline.

Regular Expression →Finding texts of same pattern

Word Net(Lexical Dictionary) →Unlike common dictionaries, Word Net is like a Lexical Dictionary i.e., here , words are stored here in an organized manner on the basis of it’s relations with other words .

Open Minded Common Sense →Common sensical facts are stored

Advantages :

1. Quick & accurate
2. Still Valid

M.L Methods :

All the major issues present in Heuristic methods like when there is some open ended issues .So, the major advantage of ML models over Heuristic method is it solves open ended problems.

Algorithms :

1. Naive-Bayes
2. Logistic Regression
3. Support Vector Machine
4. LDA (for Topic Modelling)
5. Hidden Markov Models

D.L Methods :

One of the main issues present in Machine Learning approach was that ML models can’t read texts sequentially. But in Deep Learning approach text data is read in sequentially manner and unlike ML approach it can also automate feature generation.

Algorithms :

1. RNN
2. LSTM (Long Short Time Memory)
3. GRU (Grated Recurrent Unit)
4. CNN
5. Transformers
6. Auto encoders

In RNN the main issue is it can’t process a Long Sentence(Natural Language). This issue gets resolved in LSTM algorithm. GRU is mainly used for Text Generation. Transformers revolutionized the NLP . Transformers can provide more attention to certain words . Auto Encoders are mainly used using Two Neural Networks (LSTM based) one of which acts as an encoder & other one acts as decoder.

Challenges in NLP:

1. Ambiguity
2. Contextual Words
3. Colloquialisms & slangs
4. Synonyms
5. Irony, Sarcasm & tonal difference
6. Spelling Errors
7. Creativity
8. Diversity