Language is messy. Sarcasm, ambiguity, and context trip up naive programs. Natural language processing builds systems that parse, interpret, and generate human language, from spell checkers to translation engines to chatbots.

Pipeline tasks include tokenization, part-of-speech tagging, parsing, and named entity recognition. Higher-level tasks include sentiment analysis, summarization, question answering, and dialogue. Transformers now handle most of these end-to-end.

Core NLP tasks

• Tokenization and normalization
• Part-of-speech tagging
• Named entity recognition
• Sentiment analysis
• Machine translation
• Summarization and question answering

Evaluation is contested. BLEU and ROUGE measure surface overlap, not meaning. Human evaluation is expensive and inconsistent. A model that scores well on a benchmark may still produce output that a fluent speaker finds odd.