Natural language processing (NLP) is a computational technique and a part of Soft Computing techniques for analyzing and representing naturally occurring text at one or more levels of linguistic analysis for the purpose of achieving human-like Language Processing for a variety of applications.
Natural Language Processing is a sub-field of Artificial Intelligence which deals with the methods of communicating with computer in once own language. Thus it reduces the distance between machine and human.
Natural Language Processing is a Computer Based approach of analyzing that is based on set of Technologies and set of theories.
Entire Language processing problems is partitioned into 2 tasks
- Processing of written text, using syntactic, semantic and lexical knowledge of language as well as required information of real world.
- Processing spoken language, using all the information needed plus some additional knowledge of Phonography, and some added information that can handle ambiguity that arise in speech.
Developing programs that can understand the natural language and that can comprehend visual scenes are two of the most difficult task faced by artificial intelligence researchers.
Natural language processing system should be able to
- Paraphrase an input text
- Translate the text into another language
- Draw inference from the text
- Answering question about the contents in the text
Levels of Natural Language Processing
This level deals with the interpretation of speech sound within and across words. There are 3 types of rule in these types of analysis
- Phonetic rules – For sound that is within words.
- Phonemic rules– For variations of pronunciation when words are spoken together.
- Prosodic rules – For fluctuation in intonation and stress across a sentence.
In NLP system that accept sound as input, the wanes of sounds are analysed and encoded into digitised signal for their interpretation by various rules or by comparison to the particular language model being under utilization.
This level deals with nature of words that is componential; they are composed of morpheme that is the smallest units of meaning. The meaning of each morpheme remains the same across word; human can break down an unknown word into its constituent morphemes in order to understand its meaning.
At this level, meanings of individual words are interpreted by human as well NLP system. There are several types of processing that helps in Word Level understanding. In the 1st of this being assignment of a single part of speech tagging to each word. Processing words that can function has more than one part of speech on assigned to the most appropriate part of speech tag on the context of the occurrence of things.
Now additionally at lexical level this word that have only one possible sense or meaning can be replaced by a semantic representation of that meaning.
It is a sequence of words that shows how the words related to each other. This level focuses on analysing the words in the given sentence so as to reveal the grammatical structure of the sentence. This requires grammar as well as a parser. The output of this level reveals the structural dependency and relationship between words.
Semantic processing determines the possible meaning of a sentence by focusing on the interaction among meaning in the sentence at Word Level. Thus the structures created by the syntactic analysis are assigned meaning.
The disourse level of NLP works with unit of text longer than a sentence.
This level is concerned with the useful use of language in situations and utilise context over and about the contents of the text for understanding.
Fundamental problems in Natural Language Processing System
- The functional structure of a sentence can give rise to ambiguities.
For example “I saw TajMahal flying over Agra” who is flying person or TajMahal.
- Words used by some community of people may have different meaning for a different set of people.
- Too much use of pronoun increase ambiguities.
- Conjunction used in Natural language to avoid repetition of phrase also cause NLP problems.
- Ellipse is a major problem which is hard to be implemented by NLP system.
Application of NLP System
- Information retrieval
- Information extraction
- Question answering
- Machine translation