class_factory.quiz_maker package

class_factory.quiz_maker.QuizMaker module

QuizMaker Module

The QuizMaker module offers a comprehensive framework for generating, distributing, and analyzing quiz questions based on lesson content and objectives. At its core, the QuizMaker class uses a language model (LLM) to create targeted quiz questions, ensuring these questions are relevant to the course material. This class also provides utilities for similarity checking, interactive quiz launches, and detailed results assessment.

Key Functionalities:

  1. Quiz Generation: - Automatically generates quiz questions from lesson objectives and readings. - Customizable difficulty level and quiz content based on user-provided or auto-extracted lesson objectives.

  2. Similarity Checking: - Detects overlap with previous quizzes to prevent question duplication. - Uses sentence embeddings to flag and remove questions too similar to prior quizzes.

  3. Validation and Formatting: - Validates generated questions to ensure proper format and structure. - Corrects answer formatting to meet quiz standards (e.g., answers in ‘A’, ‘B’, ‘C’, ‘D’).

  4. Saving Quizzes: - Exports quizzes as Excel files or PowerPoint presentations. - Customizes PowerPoint presentations using templates for polished quiz slides.

  5. Interactive Quiz Launch: - Launches an interactive Gradio-based web quiz for real-time participation. - Supports QR code access and real-time result saving.

  6. Results Assessment: - Analyzes and visualizes quiz results stored in CSV files. - Generates summary statistics, HTML reports, and dashboards for insights into quiz performance.

Dependencies

This module requires:

  • langchain_core: For LLM interaction and prompt handling.

  • sentence_transformers: For semantic similarity detection in quiz questions.

  • pptx: For PowerPoint presentation generation.

  • pandas: For data handling and result assessment.

  • torch: For managing device selection and embedding models.

  • gradio: For interactive quiz interfaces.

  • Custom utilities for document loading, response parsing, logging, and retry decorators.

Usage Overview

  1. Initialize QuizMaker: - Instantiate QuizMaker with required paths, lesson loader, and LLM.

  2. Generate a Quiz: - Call make_a_quiz() to create quiz questions based on lesson materials, with automatic similarity checking.

  3. Save the Quiz: - Use save_quiz() to save the quiz as an Excel file or save_quiz_to_ppt() to export to PowerPoint.

  4. Launch an Interactive Quiz: - Use launch_interactive_quiz() to start a web-based quiz, with options for real-time participation and result saving.

  5. Assess Quiz Results: - Analyze saved quiz responses with assess_quiz_results(), generating summary statistics, reports, and visualizations.

Example

from pathlib import Path
from class_factory.quiz_maker.QuizMaker import QuizMaker
from class_factory.utils.load_documents import LessonLoader
from langchain_openai import ChatOpenAI

# Set up paths and initialize components
syllabus_path = Path("/path/to/syllabus.docx")
reading_dir = Path("/path/to/lesson/readings")
project_dir = Path("/path/to/project")
llm = ChatOpenAI(api_key="your_api_key")

# Initialize lesson loader and quiz maker
lesson_loader = LessonLoader(syllabus_path=syllabus_path, reading_dir=reading_dir, project_dir=project_dir)
quiz_maker = QuizMaker(
    llm=llm,
    lesson_no=1,
    course_name="Sample Course",
    lesson_loader=lesson_loader,
    output_dir=Path("/path/to/output/dir")
)

# Generate and save a quiz
quiz = quiz_maker.make_a_quiz()
quiz_maker.save_quiz(quiz)
quiz_maker.save_quiz_to_ppt(quiz)
class class_factory.quiz_maker.QuizMaker.QuizMaker(llm, lesson_no: int, course_name: str, lesson_loader: LessonLoader, output_dir: Path | str = None, prior_quiz_path: Path | str = None, lesson_range: range = range(1, 5), quiz_prompt_for_llm: str = None, device=None, lesson_objectives: dict = None, verbose=False)[source]

Bases: BaseModel

A class to generate and manage quizzes based on lesson readings and objectives using a language model (LLM).

QuizMaker generates quiz questions from lesson content, checks for similarity with prior quizzes to avoid redundancy, and validates question format. Quizzes can be saved as Excel or PowerPoint files, launched interactively, and analyzed for performance.

llm

The language model instance for quiz generation.

syllabus_path

Path to the syllabus file.

Type:

Path

reading_dir

Directory containing lesson readings.

Type:

Path

output_dir

Directory for saving quiz files.

Type:

Path

prior_quiz_path

Directory with prior quizzes for similarity checks.

Type:

Path

lesson_range

Range of lessons for quiz generation.

Type:

range

course_name

Name of the course for context in question generation.

Type:

str

device

Device for embeddings (CPU or GPU).

rejected_questions

List of questions flagged as similar to prior quizzes.

Type:

List[Dict]

make_a_quiz(difficulty_level

int = 5, flag_threshold: float = 0.7) -> List[Dict]: Generates quiz questions with similarity checks to avoid redundancy.

save_quiz(quiz

List[Dict]) -> None: Saves quiz questions to an Excel file.

save_quiz_to_ppt(quiz

List[Dict] = None, excel_file: Path = None, template_path: Path = None) -> None: Saves quiz questions to a PowerPoint file, optionally with a template.

launch_interactive_quiz(quiz_data, sample_size

int = 5, seed: int = 42, save_results: bool = False, output_dir: Path = None, qr_name: str = None) -> None: Launches an interactive quiz using Gradio.

assess_quiz_results(quiz_data

pd.DataFrame = None, results_dir: Path = None, output_dir: Path = None) -> pd.DataFrame: Analyzes quiz results and generates summary statistics and visualizations.

Internal Methods:
_validate_llm_response(quiz_questions: Dict[str, Any], objectives: str, readings: str, prior_quiz_questions: List[str], difficulty_level: int, additional_guidance: str) -> Dict[str, Any]:

Validates generated quiz questions for relevance and format.

_validate_questions(questions: List[Dict]) -> List[Dict]:

Checks for formatting errors and corrects them.

_build_quiz_chain() -> Any:

Builds the LLM chain for quiz generation.

_load_and_merge_prior_quizzes() -> Tuple[List[str], pd.DataFrame]:

Loads and merges questions from prior quizzes for similarity checking.

_check_question_similarity(generated_questions: List[str], threshold: float = 0.6) -> List[Dict]:

Checks for question similarity against prior quizzes.

_separate_flagged_questions(questions: List[Dict], flagged_questions: List[Dict]) -> Tuple[List[Dict], List[Dict]]:

Separates flagged questions based on similarity results.

assess_quiz_results(quiz_data: DataFrame | None = None, results_dir: Path | str | None = None, output_dir: Path | str | None = None) → DataFrame[source]

Analyze quiz results, generate summary statistics, and visualize responses.

Parameters:

  • quiz_data (pd.DataFrame, optional) – DataFrame of quiz results. If None, loads results from CSV files in results_dir.

  • results_dir (Path, optional) – Directory containing CSV files of quiz results.

  • output_dir (Path, optional) – Directory for saving summary statistics and plots. Defaults to output_dir/’quiz_analysis’.

Returns:

DataFrame with summary statistics, including:

  • question (str): Question text

  • Total Responses (int): Number of unique users who answered

  • Correct Responses (int): Number of correct answers

  • Incorrect Responses (int): Number of incorrect answers

  • Percent Correct (float): Percentage of correct answers

  • Modal Answer (str): Most common answer given

Return type:

pd.DataFrame

Raises:

AssertionError – If quiz_data is provided but is not a pandas DataFrame.

launch_interactive_quiz(quiz_data: DataFrame | Path | str | List[Dict[str, Any]] = None, sample_size: int = 5, seed: int = 42, save_results: bool = False, output_dir: Path | None = None, qr_name: str | None = None) → None[source]

Launch an interactive quiz using Gradio, sampling questions from provided data or generating new data if none is provided.

Parameters:

  • quiz_data (Union[pd.DataFrame, Path, str, List[Dict[str, Any]]], optional) – Quiz questions as a DataFrame, Excel path, or list of dictionaries. If None, generates new questions.

  • sample_size (int, optional) – Number of questions to sample. Defaults to 5.

  • seed (int, optional) – Random seed for consistent sampling. Defaults to 42.

  • save_results (bool, optional) – Whether to save quiz results. Defaults to False.

  • output_dir (Path, optional) – Directory to save quiz results. Defaults to the class’s output directory.

  • qr_name (str, optional) – Name of the QR code image file for accessing the quiz.

Raises:

ValueError – If quiz_data is provided but is not a valid type.

make_a_quiz(difficulty_level: int = 5, flag_threshold: float = 0.7) → List[Dict][source]

Generate quiz questions based on lesson readings and objectives, checking for similarity with prior quizzes.

Parameters:

  • difficulty_level (int) – Difficulty of generated questions, scale 1-10. Defaults to 5.

  • flag_threshold (float) – Similarity threshold for rejecting duplicate questions. Defaults to 0.7.

Returns:

Generated quiz questions, with duplicates removed. Each dict contains:

  • question (str): The question text

  • type (str): Question type (e.g. “multiple_choice”)

    1. (str): First answer choice

    1. (str): Second answer choice

    1. (str): Third answer choice

    1. (str): Fourth answer choice

  • correct_answer (str): Letter of correct answer

Return type:

List[Dict[str, Any]]

save_quiz(quiz: List[Dict]) → None[source]

Save quiz questions to an Excel file.

Parameters:

quiz (List[Dict[str, Any]]) – List of quiz questions to save. Each dict should contain: - type (str): Question type - question (str): Question text - A) (str): First answer choice - B) (str): Second answer choice - C) (str): Third answer choice - D) (str): Fourth answer choice - correct_answer (str): Letter of correct answer

Returns:

None

save_quiz_to_ppt(quiz: List[Dict] = None, excel_file: Path | str = None, template_path: Path | str = None, filename: str = None) → None[source]

Save quiz questions to a PowerPoint presentation, with options to use a template.

Parameters:

  • quiz (List[Dict], optional) – List of quiz questions in dictionary format.

  • excel_file (Path, optional) – Path to an Excel file containing quiz questions. If provided, it overrides the quiz argument.

  • template_path (Path, optional) – Path to a PowerPoint template to apply to the generated slides.

Raises:

ValueError – If neither quiz nor excel_file is provided.

Creates a PowerPoint presentation with each question on a slide, followed by the answer slide.