# Few-Shot Learning Optimizers Overview

Welcome to our lesson on **Automatic Few-Shot Learning with DSPy**! In the previous lesson, we introduced the concept of optimization in DSPy and explored the three main categories of optimizers: Few-Shot Learning, Instruction Optimization, and Finetuning. Now, we'll dive deeper into the first category: Few-Shot Learning optimizers.

As you may recall, _few-shot learning_ is a technique where we provide the language model with examples of the task before asking it to solve a new instance. This approach helps the model understand what we're asking for and improves its performance. While you could manually select and include examples in your prompts, DSPy's few-shot optimizers automate this process, finding the most effective examples to include.

In this lesson, we'll explore four different few-shot optimizers:

1. **LabeledFewShot**: The simplest approach, which randomly selects examples from your training data.
2. **BootstrapFewShot**: A more advanced approach that generates new examples using your program itself.
3. **BootstrapFewShotWithRandomSearch**: Extends `BootstrapFewShot` by exploring multiple sets of examples to find the best combination.
4. **KNNFewShot**: A retrieval-based approach that selects examples most similar to the current input.

Each optimizer has its strengths and is suited for different scenarios. If you have very few examples (around 10), `BootstrapFewShot` is a good starting point. With more data (50+ examples), `BootstrapFewShotWithRandomSearch` can yield better results. `KNNFewShot` is particularly useful when the relevance of examples varies significantly depending on the input.

Let's explore each of these optimizers in detail, with practical examples to help you understand how to implement them in your own projects.

## LabeledFewShot: Basic Example Selection

The simplest few-shot optimizer in DSPy is `LabeledFewShot`. This optimizer takes examples from your training data and includes them in the prompt sent to the language model. It's straightforward but effective, especially when you have high-quality labeled examples.

### Implementation Example

```python
from dspy.teleprompt import LabeledFewShot

# Create the optimizer with k=8 (8 examples will be included in each prompt)
labeled_fewshot_optimizer = LabeledFewShot(k=8)

# Compile your DSPy program with the optimizer
your_dspy_program_compiled = labeled_fewshot_optimizer.compile(
    student=your_dspy_program,
    trainset=trainset
)
```

In this example, we create a `LabeledFewShot` optimizer that will include 8 examples in each prompt. The `k` parameter controls the number of examples, and you can adjust it based on your needs and the context window size of your language model.

## BootstrapFewShot: Self-Generated Examples

While `LabeledFewShot` simply uses examples from your training data, `BootstrapFewShot` goes a step further by generating additional examples using your program itself. This is particularly useful when you have limited labeled data or when you want to create more diverse examples.

### Implementation Example

```python
from dspy.teleprompt import BootstrapFewShot

# Create the optimizer
fewshot_optimizer = BootstrapFewShot(
    metric=your_defined_metric,
    max_bootstrapped_demos=4,
    max_labeled_demos=16,
    max_rounds=1,
    max_errors=5
)

# Compile your DSPy program with the optimizer
your_dspy_program_compiled = fewshot_optimizer.compile(
    student=your_dspy_program,
    trainset=trainset
)
```

## BootstrapFewShotWithRandomSearch: Finding Optimal Example Sets

Building on `BootstrapFewShot`, the `BootstrapFewShotWithRandomSearch` optimizer adds another layer of optimization by exploring multiple sets of examples to find the best combination. This is particularly useful when you have a larger training set and want to find the most effective subset of examples.

### Implementation Example

```python
from dspy.teleprompt import BootstrapFewShotWithRandomSearch

# Configure the optimizer
config = dict(
    max_bootstrapped_demos=4,
    max_labeled_demos=4,
    num_candidate_programs=10,
    num_threads=4
)

# Create the optimizer
teleprompter = BootstrapFewShotWithRandomSearch(
    metric=YOUR_METRIC_HERE,
    **config
)

# Compile your DSPy program with the optimizer
optimized_program = teleprompter.compile(
    YOUR_PROGRAM_HERE,
    trainset=YOUR_TRAINSET_HERE
)
```

## KNNFewShot: Context-Aware Example Selection

The final few-shot optimizer we'll explore is `KNNFewShot`, which takes a different approach by selecting examples based on their similarity to the current input. This is particularly useful when the relevance of examples varies significantly depending on the input.

### Implementation Example

```python
from sentence_transformers import SentenceTransformer
from dspy import Embedder
from dspy.teleprompt import KNNFewShot
from dspy import ChainOfThought

# Create an embedder using SentenceTransformer
embedder = Embedder(SentenceTransformer("all-MiniLM-L6-v2").encode)

# Create the optimizer
knn_optimizer = KNNFewShot(
    k=3,
    trainset=trainset,
    vectorizer=embedder
)

# Compile your DSPy program with the optimizer
qa_compiled = knn_optimizer.compile(
    student=ChainOfThought("question -> answer")
)
```

## Summary and Practice Preview

In this lesson, we've explored four different few-shot optimizers in DSPy:

When deciding which optimizer to use, consider the following guidelines:
- If you have very few examples (around 10), start with `BootstrapFewShot`.
- If you have more data (50+ examples), try `BootstrapFewShotWithRandomSearch`.
- If different inputs benefit from different types of examples, consider `KNNFewShot`.
- If you're just getting started and want a simple baseline, `LabeledFewShot` is a good choice.