Welcome to my notes on Natural Language Processing.
the why and what
Before we dive into Natural Language Processing, let’s understand why you should care about it in the first place.
Natural Language Processing is a fascinating intersection of computer science, artificial intelligence, and linguistics. It’s about teaching computers to understand, interpret, and respond to human language in a valuable and meaningful way.
It’s useful for
- Building conversational interfaces
- Analyzing/generating many varieties of text
- Understanding linguistics from a new perspective
things you can do with this
If you’re anything like me, you’d love to learn this for no reason- but it’s good to know what skills you can expect to learn with this content.
With the material in these pages, you should be able to build
- Sophisticated chatbots that can understand and respond to human queries
- Advanced text analysis tools that can extract sentiment, categorize content, and detect key topics
- Understand how consumer applications like ChatGPT were built
the content
I’ve divided the content into the following sections
- Linguistics of Language Models: understanding the fundamental building blocks of language
- Word Vectors: how do we represent words?
- Dependency Parsing: analyzing the dependency structure of language
- Constituency Parsing: Analyzing the hierarchical structure of sentences into nested syntactic components
- Coreference Resolution:
- Linguistics of Language Models: What do language models know about linguistics? How can we find out?
- Architectures: which architectures are most useful for nlp tasks?
- Language Models: recurrent models like LSTMs, GRU, etc.
- ConvNets + TRNNs
- Attention + Transformer: How can we apply attention and the transformer architecture to natural language tasks?
- Pretraining: everything you’ve ever wanted to know about pretraining language models
- Applications: what can we do with NLP? how should we approach these problems?
- Natural Language Generation:
- Question Answering:
- Code Generation:
- Multimodal: some fun exploration into multimodal applications