Procoder

Top 3 Winners:

Mayuri Chaudhari, Eshan Chattaraj, Vaibhav Meena

Congratulations!!

1. What does embedding represent?

• A. A compressed version of raw data used for storage

• B. A numerical representation that captures the meaning of the original data

• C. A rule-based description of text

• D. A visualization of data for humans

✔ Correct Answer: B. A numerical representation that captures the meaning of the original data

2. Which of the following best describes the structure of an embedding?

• A. A table

• B. A graph

• C. A vector

• D. A sentence

✔ Correct Answer: C. A vector

3. An embedding algorithm is considered good when:

• A. It produces smaller vectors

• B. It runs faster than other algorithms

• C. More-similar texts have closer embeddings

• D. It only works for technical text

✔ Correct Answer: C. More-similar texts have closer embeddings

4. Which pair of sentences should have embeddings that are closer to each other?

• A. “The cat sits on a mat” and “AI research is super fun”

• B. “The cat sits on a mat” and “The dog plays on the grass”

• C. “AI research is super fun” and “The dog plays on the grass”

• D. All pairs should be equally close

✔ Correct Answer: B. “The cat sits on a mat” and “The dog plays on the grass”

5. Which of the following is NOT a common use of embeddings?

• A. Classification

• B. RAG

• C. Recommender systems

• D. Hardware acceleration

✔ Correct Answer: D. Hardware acceleration

6. Embeddings can be created for which types of data?

• A. Only text

• B. Only images and audio

• C. Only numerical data

• D. Any type of data

✔ Correct Answer: D. Any type of data

7. What is the approach called when AI is used to evaluate the output of another AI?

• A. Reinforcement learning

• B. AI as a judge (or LLM as a judge)

• C. Human-in-the-loop learning

• D. Model distillation

✔ Correct Answer: B. AI as a judge (or LLM as a judge)

8. Which of the following evaluation approaches can AI be used for?

• A. Evaluating a response by itself

• B. Comparing a response to reference data

• C. Comparing one response to another response

• D. All of the above

✔ Correct Answer: D. All of the above

9. AI judges tend to work better with which type of scoring system?

• A. Large continuous numerical ranges

• B. Complex multi-step calculations

• C. Classification-based scoring

• D. Randomized scoring

✔ Correct Answer: C. Classification-based scoring

10. Which statement about tools like MLflow, Ragas, and LlamaIndex is true?

• A. They all use identical instructions and scoring systems

• B. They do not support AI-based evaluation

• C. They include faithfulness metrics, but with different instructions and scoring systems

• D. They rely only on human evaluators

✔ Correct Answer: C. They include faithfulness metrics, but with different instructions and scoring systems

11. Which of the following should be clearly explained in a judge’s prompt?

• A. The task, evaluation criteria, and scoring system

• B. Only the scoring system

• C. Only the task

• D. Only the evaluation criteria

✔ Correct Answer: A. The task, evaluation criteria, and scoring system

12. Which statement about using stronger or weaker models as judges is correct?

• A. Only weaker models can be judges

• B. Only stronger models can be judges

• C. Both stronger and weaker models can act as judges

• D. The judge must always be human

✔ Correct Answer: C. Both stronger and weaker models can act as judges

13. Imagine each sentence is turned into a point on a map using embeddings. What does it mean if two points are very close together on this map?

• A. The sentences were typed by the same person

• B. The sentences were written on the same day

• C. The sentences have the same number of words

• D. The sentences have similar meanings

✔ Correct Answer: D. The sentences have similar meanings

14. Why do people sometimes use an LLM as a judge instead of asking a single LLM answer directly and trusting it?

• A. Because judging is faster than generating any answers

• B. Because judging multiple answers can reduce the impact of a single bad response

• C. Because a judge model always knows the absolute truth

• D. Because using a judge makes the model run without electricity

✔ Correct Answer: B. Because judging multiple answers can reduce the impact of a single bad response

15. How can embeddings and an LLM as a judge work together in a question-answering system?

• A. The LLM creates embeddings, and nothing else happens

• B. They cannot be used together in the same system

• C. Embeddings write the answer, and the LLM only checks spelling

• D. Embeddings find related information, then the LLM judges which answer is best

✔ Correct Answer: D. Embeddings find related information, then the LLM judges which answer is best

For any queries, please write to capdev@harbingergroup.com