Helion


Szczegóły ebooka

Advanced Natural Language Processing with TensorFlow 2

Advanced Natural Language Processing with TensorFlow 2


In the last couple of years, there have been tremendous advances in natural language processing, and we are now moving from research labs into practical applications. Advanced Natural Language Processing with TensorFlow 2 comes with a perfect blend of both the theoretical and practical aspects of trending and complex NLP techniques.

This book is focused on innovative applications in the field of NLP, language generation, and dialogue systems. It goes into the details of applying the concepts of text pre-processing using techniques such as tokenization, parts of speech tagging, and lemmatization using popular libraries such as Stanford NLP and SpaCy. Named Entity Recognition (NER), a cornerstone of task-oriented bots, is built from scratch using Conditional Random Fields and Viterbi Decoding on top of RNNs.

Taking a practical and application-focused perspective, the book covers key emerging areas such as generating text for use in sentence completion and text summarization, bridging images and text by generating captions for images, and managing dialogue aspects of chatbot design. It also covers one of the most important reasons behind recent advances in NLP - applying transfer learning and fine-tuning using TensorFlow 2.

Further, it covers practical techniques that can simplify the labelling of textual data which otherwise proves to be a costly affair. The book also has a working code for each tech piece so that you can adapt them to your use cases.

By the end of this TensorFlow book, you will have an advanced knowledge of the tools, techniques and deep learning architecture used to solve complex NLP problems.

  • Preface
    • Who this book is for
    • What this book covers
    • To get the most out of this book
      • Download the example code files
      • Download the color images
      • Conventions used
    • Get in touch
      • Reviews
  • Essentials of NLP
    • A typical text processing workflow
    • Data collection and labeling
      • Collecting labeled data
        • Development environment setup
      • Enabling GPUs on Google Colab
    • Text normalization
      • Modeling normalized data
      • Tokenization
        • Segmentation in Japanese
        • Modeling tokenized data
      • Stop word removal
        • Modeling data with stop words removed
      • Part-of-speech tagging
        • Modeling data with POS tagging
      • Stemming and lemmatization
    • Vectorizing text
      • Count-based vectorization
        • Modeling after count-based vectorization
      • Term Frequency-Inverse Document Frequency (TF-IDF)
        • Modeling using TF-IDF features
      • Word vectors
        • Pretrained models using Word2Vec embeddings
    • Summary
  • Understanding Sentiment in Natural Language with BiLSTMs
    • Natural language understanding
    • Bi-directional LSTMs BiLSTMs
      • RNN building blocks
      • Long short-term memory (LSTM) networks
      • Gated recurrent units (GRUs)
      • Sentiment classification with LSTMs
        • Loading the data
        • Normalization and vectorization
        • LSTM model with embeddings
        • BiLSTM model
    • Summary
  • Named Entity Recognition (NER) with BiLSTMs, CRFs, and Viterbi Decoding
    • Named Entity Recognition
      • The GMB data set
    • Loading the data
    • Normalizing and vectorizing data
    • A BiLSTM model
    • Conditional random fields (CRFs)
    • NER with BiLSTM and CRFs
      • Implementing the custom CRF layer, loss, and model
        • A custom CRF model
        • A custom loss function for NER using a CRF
      • Implementing custom training
    • Viterbi decoding
      • The probability of the first word label
    • Summary
  • Transfer Learning with BERT
    • Transfer learning overview
      • Types of transfer learning
        • Domain adaptation
        • Multi-task learning
        • Sequential learning
    • IMDb sentiment analysis with GloVe embeddings
      • GloVe embeddings
      • Loading IMDb training data
      • Loading pre-trained GloVe embeddings
      • Creating a pre-trained embedding matrix using GloVe
      • Feature extraction model
      • Fine-tuning model
    • BERT-based transfer learning
      • Encoder-decoder networks
      • Attention model
      • Transformer model
      • The bidirectional encoder representations from transformers (BERT) model
        • Tokenization and normalization with BERT
        • Pre-built BERT classification model
        • Custom model with BERT
    • Summary
  • Generating Text with RNNs and GPT-2
    • Generating text one character at a time
      • Data loading and pre-processing
      • Data normalization and tokenization
      • Training the model
      • Implementing learning rate decay as custom callback
      • Generating text with greedy search
    • Generative Pre-Training (GPT-2) model
      • Generating text with GPT-2
    • Summary
  • Text Summarization with Seq2seq Attention and Transformer Networks
    • Overview of text summarization
    • Data loading and pre-processing
    • Data tokenization and vectorization
    • Seq2seq model with attention
      • Encoder model
      • Bahdanau attention layer
      • Decoder model
    • Training the model
    • Generating summaries
      • Greedy search
      • Beam search
      • Decoding penalties with beam search
    • Evaluating summaries
    • ROUGE metric evaluation
    • Summarization state of the art
    • Summary
  • Multi-Modal Networks and Image Captioning with ResNets and Transformer Networks
    • Multi-modal deep learning
      • Vision and language tasks
    • Image captioning
    • MS-COCO dataset for image captioning
    • Image processing with CNNs and ResNet50
      • CNNs
        • Convolutions
        • Pooling
        • Regularization with dropout
        • Residual connections and ResNets
    • Image feature extraction with ResNet50
    • The Transformer model
      • Positional encoding and masks
      • Scaled dot-product and multi-head attention
      • VisualEncoder
      • Decoder
      • Transformer
    • Training the Transformer model with VisualEncoder
      • Loading training data
      • Instantiating the Transformer model
      • Custom learning rate schedule
      • Loss and metrics
      • Checkpoints and masks
      • Custom training
    • Generating captions
    • Improving performance and state-of-the-art models
    • Summary
  • Weakly Supervised Learning for Classification with Snorkel
    • Weak supervision
      • Inner workings of weak supervision with labeling functions
    • Using weakly supervised labels to improve IMDb sentiment analysis
      • Pre-processing the IMDb dataset
      • Learning a subword tokenizer
      • A BiLSTM baseline model
        • Tokenization and vectorizing data
        • Training using a BiLSTM model
    • Weakly supervised labeling with Snorkel
      • Iterating on labeling functions
    • Nave-Bayes model for finding keywords
      • Evaluating weakly supervised labels on the training set
      • Generating unsupervised labels for unlabeled data
      • Training BiLSTM on weakly supervised data from Snorkel
    • Summary
  • Building Conversational AI Applications with Deep Learning
    • Overview of conversational agents
      • Task-oriented or slot-filling systems
    • Question-answering and MRC conversational agents
    • General conversational agents
    • Summary
    • Epilogue
  • Installation and Setup Instructions for Code
    • GitHub location
    • Chapter 1 installation instructions
    • Chapter 2 installation instructions
    • Chapter 3 installation instructions
    • Chapter 4 installation instructions
    • Chapter 5 installation instructions
    • Chapter 6 installation instructions
    • Chapter 7 installation instructions
    • Chapter 8 installation instructions
    • Chapter 9 installation instructions
  • Other Books You May Enjoy
  • Index