プロンプト設計の確認問題

正解: Chain-of-thought prompting

Chain-of-thought (CoT) prompting encourages the model to produce intermediate reasoning steps, which improves accuracy on complex tasks. The stem asks: Which technique instructs the model to reason step-by-step before giving a final answer?. The keyed answer is 2. Options: Zero-shot prompting; Chain-of-thought prompting; Retrieval-Augmented Generation; Fine-tuning. Keep this in the prompt_engineering bucket, and reject the neighbor term that solves a different job.

正解: The randomness of the output distribution

Temperature scales the logits before softmax sampling. Higher values produce more random outputs; lower values make outputs more deterministic. The stem asks: What does 'temperature' control in LLM generation?. The keyed answer is 2. Options: The number of tokens generated; The randomness of the output distribution; The length of the system prompt; The number of attention heads used. Keep this in the prompt_engineering bucket, and reject the neighbor term that solves a different job.

正解: To demonstrate the desired input-output pattern

Few-shot examples show the model the expected format and behavior without updating any weights, relying solely on in-context learning. The stem asks: In few-shot prompting, what is the primary purpose of the examples provided?. The keyed answer is 3. Options: To fine-tune the model weights; To increase the context window size; To demonstrate the desired input-output pattern; To reduce hallucination by adding retrieval. Keep this in the prompt_engineering bucket, and reject the neighbor term that solves a different job.

正解: Greedy decoding

Greedy decoding picks the single most probable token at every step, which is fast but can lead to repetitive or suboptimal outputs. The stem asks: Which sampling strategy always selects the highest-probability token at each step?. The keyed answer is 3. Options: Top-p sampling; Top-k sampling; Greedy decoding; Beam search. Keep this in the prompt_engineering bucket, and reject the neighbor term that solves a different job.

正解: It sets persistent context and instructions for the entire conversation

The system prompt is prepended before user turns and allows developers to define the model's persona, constraints, and task context without repeating instructions each turn. The stem asks: What is the main advantage of a system prompt over a user prompt in chat-based LLM APIs?. The keyed answer is 2. Options: It is processed faster by the model; It sets persistent context and instructions for the entire conversation; It allows the model to access external tools; It increases the maximum token limit. Keep this in the prompt_engineering bucket, and reject the neighbor term that solves a different job.

正解: Hallucination

Hallucination refers to the model producing plausible-sounding but incorrect or fabricated content, a key challenge in production LLM systems. The stem asks: Which phenomenon occurs when an LLM confidently generates factually incorrect information?. The keyed answer is 2. Options: Overfitting; Hallucination; Catastrophic forgetting; Gradient explosion. Keep this in the prompt_engineering bucket, and reject the neighbor term that solves a different job.

正解: Samples from the smallest set of tokens whose cumulative probability exceeds p

Top-p sampling dynamically adjusts the candidate token pool based on cumulative probability mass, balancing diversity and coherence better than fixed top-k. The stem asks: What does 'top-p' (nucleus) sampling do?. The keyed answer is 2. Options: Selects the top p% of layers to activate; Samples from the smallest set of tokens whose cumulative probability exceeds p; Limits output to exactly p tokens; Penalizes tokens that have appeared more than p times. Keep this in the prompt_engineering bucket, and reject the neighbor term that solves a different job.

正解: Instructing the model to adopt a specific persona or expert role

Role prompting (e.g. 'You are an expert Python developer') primes the model to respond with vocabulary and reasoning patterns consistent with that role. The stem asks: In prompt engineering, what is 'role prompting'?. The keyed answer is 2. Options: Assigning a numeric role ID to each token; Instructing the model to adopt a specific persona or expert role; Rotating system prompts across multiple turns; Using RLHF to reinforce preferred roles. Keep this in the prompt_engineering bucket, and reject the neighbor term that solves a different job.

正解: Retrieval-Augmented Generation (RAG)

RAG retrieves relevant passages from a knowledge base and injects them into the prompt, allowing the model to cite factual sources rather than rely on parametric memory. The stem asks: Which method reduces hallucination by grounding LLM responses in retrieved external docume. The keyed answer is 2. Options: Prompt chaining; Retrieval-Augmented Generation (RAG); Self-consistency decoding; Instruction tuning. Keep this in the prompt_engineering bucket, and reject the neighbor term that solves a different job.

正解: A malicious input that overrides or hijacks the original prompt instructions

Prompt injection is a security vulnerability where adversarial user input manipulates the model into ignoring developer instructions, potentially leaking data or performing unintended actions. The stem asks: What is 'prompt injection'?. The keyed answer is 3. Options: Adding dynamic variables into a prompt template; A technique to improve prompt diversity; A malicious input that overrides or hijacks the original prompt instructions; Injecting retrieved documents into the context window. Keep this in the prompt_engineering bucket, and reject the neighbor term that solves a different job.

正解: Self-consistency

Self-consistency samples multiple reasoning chains and takes a majority vote over the final answers, significantly improving accuracy on reasoning-heavy tasks. The stem asks: Which decoding technique generates multiple outputs and selects the most consistent answer. The keyed answer is 3. Options: Beam search; Greedy decoding; Self-consistency; Top-k sampling. Keep this in the prompt_engineering bucket, and reject the neighbor term that solves a different job.

正解: To constrain the model's response to a specific structure such as JSON or markdown

Specifying the output format (e.g. 'Respond only in valid JSON') makes downstream parsing reliable and reduces free-form responses that are harder to process programmatically. The stem asks: What is the primary purpose of 'output format instructions' in a prompt?. The keyed answer is 2. Options: To reduce the model's token budget; To constrain the model's response to a specific structure such as JSON or markdown; To prevent the model from using certain words; To enable streaming responses. Keep this in the prompt_engineering bucket, and reject the neighbor term that solves a different job.

正解: The maximum number of tokens the model can process in a single forward pass

The context window defines the total tokens (prompt + completion) the model can attend to at once. Exceeding it truncates input or causes errors. The stem asks: What does 'context window' refer to in LLMs?. The keyed answer is 2. Options: The graphical interface of the prompt editor; The maximum number of tokens the model can process in a single forward pass; The number of examples used during fine-tuning; The sliding window used in convolutional layers. Keep this in the prompt_engineering bucket, and reject the neighbor term that solves a different job.

正解: Add explicit step-by-step instructions and output format constraints

Explicit instructions reduce ambiguity, which is the primary cause of inconsistent outputs. Structured prompts leave less room for the model to interpret the task differently each run. The stem asks: Which strategy is most effective when a prompt produces inconsistent results across runs?. The keyed answer is 3. Options: Increase the temperature; Switch to a smaller model; Add explicit step-by-step instructions and output format constraints; Remove the system prompt. Keep this in the prompt_engineering bucket, and reject the neighbor term that solves a different job.

正解: Breaking a complex task into sequential prompts where each output feeds the next

Prompt chaining decomposes difficult tasks into manageable subtasks. Each LLM call handles one step, and its output becomes input for the next, improving accuracy and controllability. The stem asks: What is 'prompt chaining'?. The keyed answer is 2. Options: Linking multiple API keys together for load balancing; Breaking a complex task into sequential prompts where each output feeds the next; Repeating the same prompt until the model converges; Encoding prompts as embeddings and chaining them in vector space. Keep this in the prompt_engineering bucket, and reject the neighbor term that solves a different job.

正解: max_tokens

The max_tokens parameter caps the number of tokens in the completion. Setting it appropriately avoids truncated responses and unnecessary API costs. The stem asks: Which parameter directly controls the maximum length of the model's response?. The keyed answer is 3. Options: temperature; top_p; max_tokens; frequency_penalty. Keep this in the prompt_engineering bucket, and reject the neighbor term that solves a different job.

正解: The model is penalized for repeating tokens that have already appeared

frequency_penalty reduces repetition by subtracting a penalty proportional to how often a token has appeared so far, encouraging more varied vocabulary. The stem asks: What is the effect of a high 'frequency_penalty' value in OpenAI-compatible APIs?. The keyed answer is 2. Options: The model generates responses faster; The model is penalized for repeating tokens that have already appeared; The model focuses only on high-frequency training data; The model increases output diversity by raising temperature. Keep this in the prompt_engineering bucket, and reject the neighbor term that solves a different job.

正解: Zero-shot provides no examples; few-shot includes one or more demonstrations

In zero-shot prompting the model must generalize from instructions alone. Few-shot adds labeled examples in the prompt to guide format and reasoning without weight updates. The stem asks: What distinguishes 'zero-shot' from 'few-shot' prompting?. The keyed answer is 2. Options: Zero-shot uses a larger model; few-shot uses a smaller one; Zero-shot provides no examples; few-shot includes one or more demonstrations; Zero-shot sets temperature to 0; few-shot uses sampling; Zero-shot requires fine-tuning; few-shot does not. Keep this in the prompt_engineering bucket, and reject the neighbor term that solves a different job.

正解: Self-refinement / self-critique

Self-refinement prompts the model to review its draft answer for errors or improvements, then produce a revised response—an iterative loop that improves output quality without human feedback. The stem asks: Which technique asks the model to verify or critique its own initial response before final. The keyed answer is 3. Options: Prompt injection; Retrieval-Augmented Generation; Self-refinement / self-critique; Beam search. Keep this in the prompt_engineering bucket, and reject the neighbor term that solves a different job.

正解: A unit of text (roughly a word or subword) that the model processes

Tokenizers split input text into tokens before feeding it to the model. Tokens are subword units; for example, 'unhappiness' might split into ['un','happiness']. Billing and context limits are measured in tokens. The stem asks: In the context of LLM APIs, what is a 'token'?. The keyed answer is 2. Options: An API authentication credential; A unit of text (roughly a word or subword) that the model processes; A single parameter in the model's weight matrix; A unique identifier for each API request. Keep this in the prompt_engineering bucket, and reject the neighbor term that solves a different job.