Natural Language Processing - MCQ Practice Questions

A standard NLP pipeline begins with Tokenization (splitting text into tokens), followed by POS Tagging (assigning grammatical tags), then Parsing (analyzing grammatical structure), and finally Named Entity Recognition (identifying entities like names and places). This sequential order ensures each step builds on the output of the previous one.

Perplexity is a standard metric for evaluating language models. It measures how well a probability distribution predicts a sample. Mathematically, it is the exponentiated average negative log-likelihood of the test data. A lower perplexity indicates a better model, as the model is less 'surprised' by the test data.

Word2Vec offers two architectures. The Skip-gram model takes a center (target) word as input and tries to predict the surrounding context words. Conversely, the CBOW model takes the surrounding context words as input and predicts the center word. GloVe and FastText are separate embedding techniques altogether.

The attention mechanism, as described in 'Attention is All You Need' (Vaswani et al., 2017), enables the model to dynamically focus on different parts of the input sequence when generating each token of the output. This allows the model to capture long-range dependencies effectively, unlike RNNs which struggle with distant context. The Transformer architecture built on this mechanism replaced recurrence entirely.

Stemming is a crude heuristic process that chops off word endings to approximate a root (e.g., 'running' → 'run', 'studies' → 'studi'). The result may not be a real word. Lemmatization, on the other hand, uses vocabulary and morphological analysis to return the proper base or dictionary form (lemma) of a word (e.g., 'studies' → 'study'). Lemmatization is more accurate but computationally more expensive.

IDF (Inverse Document Frequency) is computed as the logarithm of the ratio of total documents to the number of documents containing the term. A high IDF score means the term appears in very few documents, indicating it is rare and potentially carries significant discriminative information. Common words like 'the' or 'is' appear in almost every document, resulting in a very low IDF score.

A sequence-to-sequence (Seq2Seq) model maps an input sequence to an output sequence, where both sequences can have different lengths. Machine Translation (e.g., translating an English sentence to French) is a classic Seq2Seq task. Sentiment Analysis and Text Classification map a sequence to a single label, while Named Entity Recognition maps each token to a label (sequence labeling), not a new sequence.

BERT uses the Masked Language Model (MLM) objective as one of its two pre-training tasks. A random 15% of input tokens are masked (replaced with a [MASK] token), and the model is trained to predict the original identity of these masked words using the bidirectional context from all surrounding tokens. This forces the model to learn deep bidirectional representations. The other pre-training task is Next Sentence Prediction (NSP).

Shift-Reduce parsing is a bottom-up parsing technique. It uses a stack and an input buffer. In the 'shift' operation, the next input token is pushed onto the stack. In the 'reduce' operation, a sequence of tokens on the stack that matches the right-hand side of a grammar rule is replaced by the rule's left-hand side symbol. This continues until the entire input is reduced to the start symbol. Top-Down and Recursive Descent parsers begin from the root and work downward.

The Bag-of-Words model represents a document as a vector of word frequencies, completely disregarding word order and grammatical context. For example, 'dog bites man' and 'man bites dog' would have identical BoW representations. This loss of sequential and semantic information is a fundamental limitation. It also suffers from high dimensionality and sparsity. Dense embeddings like Word2Vec address the semantic relationship limitation.