class_factory.quiz_maker package

Submodules

class_factory.quiz_maker.QuizMaker module

QuizMaker Module

The QuizMaker module offers a comprehensive framework for generating, distributing, and analyzing quiz questions based on lesson content and objectives. At its core, the QuizMaker class uses a language model (LLM) to create targeted quiz questions, ensuring these questions are relevant to the course material. This class also provides utilities for similarity checking, interactive quiz launches, and detailed results assessment.

Key Functionalities:

  1. Quiz Generation: - Automatically generates quiz questions from lesson objectives and readings. - Customizable difficulty level and quiz content based on user-provided or auto-extracted lesson objectives.

  2. Similarity Checking: - Detects overlap with previous quizzes to prevent question duplication. - Uses sentence embeddings to flag and remove questions too similar to prior quizzes.

  3. Validation and Formatting: - Validates generated questions to ensure proper format and structure. - Corrects answer formatting to meet quiz standards (e.g., answers in ‘A’, ‘B’, ‘C’, ‘D’).

  4. Saving Quizzes: - Exports quizzes as Excel files or PowerPoint presentations. - Customizes PowerPoint presentations using templates for polished quiz slides.

  5. Interactive Quiz Launch: - Launches an interactive Gradio-based web quiz for real-time participation. - Supports QR code access and real-time result saving.

  6. Results Assessment: - Analyzes and visualizes quiz results stored in CSV files. - Generates summary statistics, HTML reports, and dashboards for insights into quiz performance.

Dependencies

This module requires:

  • langchain_core: For LLM interaction and prompt handling.

  • sentence_transformers: For semantic similarity detection in quiz questions.

  • pptx: For PowerPoint presentation generation.

  • pandas: For data handling and result assessment.

  • torch: For managing device selection and embedding models.

  • gradio: For interactive quiz interfaces.

  • Custom utilities for document loading, response parsing, logging, and retry decorators.

Usage Overview

  1. Initialize QuizMaker: - Instantiate QuizMaker with required paths, lesson loader, and LLM.

  2. Generate a Quiz: - Call make_a_quiz() to create quiz questions based on lesson materials, with automatic similarity checking.

  3. Save the Quiz: - Use save_quiz() to save the quiz as an Excel file or save_quiz_to_ppt() to export to PowerPoint.

  4. Launch an Interactive Quiz: - Use launch_interactive_quiz() to start a web-based quiz, with options for real-time participation and result saving.

  5. Assess Quiz Results: - Analyze saved quiz responses with assess_quiz_results(), generating summary statistics, reports, and visualizations.

Example

from pathlib import Path
from class_factory.quiz_maker.QuizMaker import QuizMaker
from class_factory.utils.load_documents import LessonLoader
from langchain_openai import ChatOpenAI

# Set up paths and initialize components
syllabus_path = Path("/path/to/syllabus.docx")
reading_dir = Path("/path/to/lesson/readings")
project_dir = Path("/path/to/project")
llm = ChatOpenAI(api_key="your_api_key")

# Initialize lesson loader and quiz maker
lesson_loader = LessonLoader(syllabus_path=syllabus_path, reading_dir=reading_dir, project_dir=project_dir)
quiz_maker = QuizMaker(
    llm=llm,
    lesson_no=1,
    course_name="Sample Course",
    lesson_loader=lesson_loader,
    output_dir=Path("/path/to/output/dir")
)

# Generate and save a quiz
quiz = quiz_maker.make_a_quiz()
quiz_maker.save_quiz(quiz)
quiz_maker.save_quiz_to_ppt(quiz)
class class_factory.quiz_maker.QuizMaker.QuizMaker(llm, lesson_no: int, course_name: str, lesson_loader: LessonLoader, output_dir: Path | str = None, prior_quiz_path: Path | str = None, lesson_range: range = range(1, 5), quiz_prompt_for_llm: str = None, device=None, lesson_objectives: dict = None, verbose=False)[source]

Bases: BaseModel

A class to generate and manage quizzes based on lesson readings and objectives using a language model (LLM).

QuizMaker generates quiz questions from lesson content, checks for similarity with prior quizzes to avoid redundancy, and validates question format. Quizzes can be saved as Excel or PowerPoint files, launched interactively, and analyzed for performance.

llm

The language model instance for quiz generation.

syllabus_path

Path to the syllabus file.

Type:

Path

reading_dir

Directory containing lesson readings.

Type:

Path

output_dir

Directory for saving quiz files.

Type:

Path

prior_quiz_path

Directory with prior quizzes for similarity checks.

Type:

Path

lesson_range

Range of lessons for quiz generation.

Type:

range

course_name

Name of the course for context in question generation.

Type:

str

device

Device for embeddings (CPU or GPU).

rejected_questions

List of questions flagged as similar to prior quizzes.

Type:

List[Dict]

make_a_quiz(difficulty_level

int = 5, flag_threshold: float = 0.7) -> List[Dict]: Generates quiz questions with similarity checks to avoid redundancy.

save_quiz(quiz

List[Dict]) -> None: Saves quiz questions to an Excel file.

save_quiz_to_ppt(quiz

List[Dict] = None, excel_file: Path = None, template_path: Path = None) -> None: Saves quiz questions to a PowerPoint file, optionally with a template.

launch_interactive_quiz(quiz_data, sample_size

int = 5, seed: int = 42, save_results: bool = False, output_dir: Path = None, qr_name: str = None) -> None: Launches an interactive quiz using Gradio.

assess_quiz_results(quiz_data

pd.DataFrame = None, results_dir: Path = None, output_dir: Path = None) -> pd.DataFrame: Analyzes quiz results and generates summary statistics and visualizations.

Internal Methods:
_validate_llm_response(quiz_questions: Dict[str, Any], objectives: str, readings: str, prior_quiz_questions: List[str], difficulty_level: int, additional_guidance: str) -> Dict[str, Any]:

Validates generated quiz questions for relevance and format.

_validate_questions(questions: List[Dict]) -> List[Dict]:

Checks for formatting errors and corrects them.

_build_quiz_chain() -> Any:

Builds the LLM chain for quiz generation.

_load_and_merge_prior_quizzes() -> Tuple[List[str], pd.DataFrame]:

Loads and merges questions from prior quizzes for similarity checking.

_check_question_similarity(generated_questions: List[str], threshold: float = 0.6) -> List[Dict]:

Checks for question similarity against prior quizzes.

_separate_flagged_questions(questions: List[Dict], flagged_questions: List[Dict]) -> Tuple[List[Dict], List[Dict]]:

Separates flagged questions based on similarity results.

make_a_quiz(difficulty_level: int = 5, flag_threshold: float = 0.7) → List[Dict][source]

Generate quiz questions based on lesson readings and objectives, checking for similarity with prior quizzes.

Parameters:

  • difficulty_level (int) – Difficulty of generated questions, scale 1-10. Defaults to 5.

  • flag_threshold (float) – Similarity threshold for rejecting duplicate questions. Defaults to 0.7.

Returns:

Generated quiz questions, with duplicates removed. Each dict contains:

  • question (str): The question text

  • type (str): Question type (e.g. “multiple_choice”)

    1. (str): First answer choice

    1. (str): Second answer choice

    1. (str): Third answer choice

    1. (str): Fourth answer choice

  • correct_answer (str): Letter of correct answer

Return type:

List[Dict[str, Any]]

save_quiz(quiz: List[Dict]) → None[source]

Save quiz questions to an Excel file, standardizing answer key styles.

Parameters:

quiz (List[Dict[str, Any]]) – List of quiz questions to save. Each dict should contain: - type (str): Question type - question (str): Question text - A/B/C/D or A)/B)/C)/D) (str): Answer choices - correct_answer (str): Letter of correct answer

Returns:

None

save_quiz_to_ppt(quiz: List[Dict] = None, excel_file: Path | str = None, template_path: Path | str = None, filename: str = None) → None[source]

Save quiz questions to a PowerPoint presentation, with options to use a template.

quiz_questions = response.model_dump() if hasattr(response, ‘model_dump’) else json.loads(response.replace(’```json

‘, ‘’).replace(’ ```’, ‘’)) if isinstance(response, str) else response

quiz (List[Dict], optional): List of quiz questions in dictionary format. excel_file (Path, optional): Path to an Excel file containing quiz questions. If provided, it overrides the quiz argument. template_path (Path, optional): Path to a PowerPoint template to apply to the generated slides.

Raises:

ValueError: If neither quiz nor excel_file is provided.

Creates a PowerPoint presentation with each question on a slide, followed by the answer slide.

launch_interactive_quiz(quiz_data: DataFrame | Path | str | List[Dict[str, Any]] = None, sample_size: int = 5, seed: int = 42, save_results: bool = False, output_dir: Path | None = None, qr_name: str | None = None) → None[source]

Launch an interactive quiz using Gradio, sampling questions from provided data or generating new data if none is provided.

Parameters:

  • quiz_data (Union[pd.DataFrame, Path, str, List[Dict[str, Any]]], optional) – Quiz questions as a DataFrame, Excel path, or list of dictionaries. If None, generates new questions.

  • sample_size (int, optional) – Number of questions to sample. Defaults to 5.

  • seed (int, optional) – Random seed for consistent sampling. Defaults to 42.

  • save_results (bool, optional) – Whether to save quiz results. Defaults to False.

  • output_dir (Path, optional) – Directory to save quiz results. Defaults to the class’s output directory.

  • qr_name (str, optional) – Name of the QR code image file for accessing the quiz.

Raises:

ValueError – If quiz_data is provided but is not a valid type.

assess_quiz_results(quiz_data: DataFrame | None = None, results_dir: Path | str | None = None, output_dir: Path | str | None = None) → DataFrame[source]

Analyze quiz results, generate summary statistics, and visualize responses.

Parameters:

  • quiz_data (pd.DataFrame, optional) – DataFrame of quiz results. If None, loads results from CSV files in results_dir.

  • results_dir (Path, optional) – Directory containing CSV files of quiz results.

  • output_dir (Path, optional) – Directory for saving summary statistics and plots. Defaults to output_dir/’quiz_analysis’.

Returns:

DataFrame with summary statistics, including:

  • question (str): Question text

  • Total Responses (int): Number of unique users who answered

  • Correct Responses (int): Number of correct answers

  • Incorrect Responses (int): Number of incorrect answers

  • Percent Correct (float): Percentage of correct answers

  • Modal Answer (str): Most common answer given

Return type:

pd.DataFrame

Raises:

AssertionError – If quiz_data is provided but is not a pandas DataFrame.

class_factory.quiz_maker.quiz_prompts module

quiz_prompts.py

This module defines prompt templates for generating quizzes using large language models (LLMs). It provides both system and human message templates, as well as a composite chat prompt template, to guide the LLM in creating quiz questions that align with lesson objectives, course content, and specified formatting requirements.

Key Components: - prompt_str: A detailed string template for quiz generation instructions. - quiz_prompt_system: System message template for LLM context. - quiz_prompt_human: Human message template with explicit quiz generation instructions. - quiz_prompt: A ChatPromptTemplate combining system and human messages for use with LLM chains.

Usage: Import quiz_prompt and use it as the prompt in an LLM chain for quiz question generation.

class_factory.quiz_maker.quiz_to_app module

This script creates an interactive quiz interface using Gradio based on quiz data from an Excel file. It takes the results of quiz generation (e.g., from 01_make_a_quiz.py) and transforms them into a fully interactive web-based quiz, allowing users to navigate through the questions, submit answers, and receive feedback.

Workflow: 1. Load Quiz Data: Reads quiz questions and answers from an Excel file with a comparable structure, including

multiple-choice questions, answer options (A, B, C, D), and the correct answer.

  1. Interactive Interface: - Displays one question at a time along with multiple-choice options. - Users can submit their answers and receive immediate feedback (correct/incorrect). - “Next” and “Back” buttons allow users to navigate between questions.

  2. Feedback and Navigation: For each question, feedback is provided based on the user’s input, and users can navigate forward or backward through the quiz.

  3. Customization: The interface supports custom themes and CSS styling for visual consistency and enhanced user experience.

  4. Saving and Launching: The Gradio app is launched with options to share, making it accessible for students or participants.

Dependencies: - Requires Gradio for the interactive interface and pandas for loading quiz data from an Excel file. - The quiz data file should be structured with columns for ‘question’, ‘A)’, ‘B)’, ‘C)’, ‘D)’, and ‘correct_answer’. - Ensure the necessary environment variables (e.g., syllabus_path) are set correctly for locating the input quiz file.

This script generates a fully functional quiz web app, ready for real-time user interaction and feedback.

class_factory.quiz_maker.quiz_to_app.load_data(quiz_path: Path, sample: int = None) → DataFrame[source]

Loads quiz data from an Excel file.

Parameters:

quiz_path (Path) – Path to the Excel file containing quiz questions.

Returns:

A pandas DataFrame containing the quiz data, with columns for questions,

choices (A, B, C, D), and correct answers.

Return type:

pd.DataFrame

class_factory.quiz_maker.quiz_to_app.log_quiz_result(user_id: str, question: str, user_answer: str, correct_answer: str, is_correct: bool, output_dir: Path | str = None)[source]

Logs the quiz result to a CSV file.

Parameters:

  • user_id (str) – The unique identifier for the user (can be a session ID or timestamp).

  • question (str) – The quiz question.

  • user_answer (str) – The user’s submitted answer.

  • correct_answer (str) – The correct answer for the question.

  • is_correct (bool) – Whether the user’s answer was correct.

Raises: KeyError: if no output directory specified

class_factory.quiz_maker.quiz_to_app.submit_answer(current_index: int, user_answer: str, quiz_data: DataFrame, user_id: str, save_results: bool, output_dir: Path | str = None) → str[source]

Checks the submitted answer for the current quiz question and logs the result if log is True.

Parameters:

  • current_index (int) – The index of the current question.

  • user_answer (str) – The user’s selected answer for the current question.

  • quiz_data (pd.DataFrame) – The DataFrame containing quiz questions and answers.

  • user_id (str) – A unique identifier for the user (can be a session ID or timestamp).

  • log (bool) – Whether to log the quiz result.

Returns:

Feedback indicating whether the user’s answer is correct or incorrect,

along with the correct answer if incorrect.

Return type:

str

class_factory.quiz_maker.quiz_to_app.next_question(current_index: int, quiz_data: DataFrame) → tuple[source]

Advances to the next quiz question, updating the question and choices. Hides the submit/next/back buttons when the quiz is completed.

Parameters:

  • current_index (int) – The index of the current question.

  • quiz_data (pd.DataFrame) – The DataFrame containing quiz questions and answers.

Returns:

A tuple containing:

  • gr.Radio.update: The updated question and choices to display.

  • str: Empty feedback text.

  • int: The updated current question index.

  • gr.Button.update: Update for submit button visibility.

  • gr.Button.update: Update for next button visibility.

  • gr.Button.update: Update for back button visibility.

Return type:

tuple

class_factory.quiz_maker.quiz_to_app.prev_question(current_index: int, quiz_data: DataFrame) → tuple[source]

Returns to the previous quiz question, updating the question and choices.

Parameters:

  • current_index (int) – The index of the current question.

  • quiz_data (pd.DataFrame) – The DataFrame containing quiz questions and answers.

Returns:

A tuple containing:

  • gr.Radio.update: The updated question and choices to display.

  • str: Empty feedback text.

  • int: The updated current question index.

  • gr.Button.update: Update for submit button visibility.

  • gr.Button.update: Update for next button visibility.

  • gr.Button.update: Update for back button visibility.

Return type:

tuple

class_factory.quiz_maker.quiz_to_app.quiz_app(quiz_data: DataFrame, share: bool = True, save_results: bool = True, output_dir: Path | str = None, qr_name: str = None) → None[source]

Launches an interactive quiz application using Gradio. :param quiz_data: The quiz questions and answers. :type quiz_data: pd.DataFrame :param log: Whether to log quiz results (default is True). :type log: bool

class_factory.quiz_maker.quiz_viz module

quiz_viz.py

This module provides visualization utilities for quiz results, including: - An interactive dashboard (using Dash and Plotly) to display summary statistics and per-question analysis. - Generation of static HTML reports with embedded plots and summary tables for quiz assessments.

Key Functions: - generate_dashboard: Launches a Dash web app for interactive quiz result exploration. - generate_html_report: Creates a static HTML report with summary statistics and per-question plots. - create_question_figure: Generates Plotly figures for individual quiz questions.

Dependencies: pandas, plotly, dash, jinja2 (for HTML reports).

class_factory.quiz_maker.quiz_viz.generate_dashboard(df: DataFrame, summary: DataFrame, test_mode: bool = False) → None[source]

Launch an interactive dashboard using Dash to display quiz summary statistics and per-question plots.

Parameters:

  • df (pd.DataFrame) – DataFrame containing quiz responses.

  • test_mode (bool) – If True, does not launch the server (for testing purposes).

  • summary (pd.DataFrame) – DataFrame containing summary statistics.

class_factory.quiz_maker.quiz_viz.generate_html_report(df: DataFrame, summary: DataFrame, output_dir: Path, quiz_df: DataFrame = None) → None[source]

Generate a static HTML report with summary statistics and per-question plots for quiz results.

Parameters:

  • df (pd.DataFrame) – DataFrame containing quiz responses.

  • summary (pd.DataFrame) – DataFrame containing summary statistics.

  • output_dir (Path) – Directory where the report will be saved.

  • quiz_df (pd.DataFrame, optional) – DataFrame containing the quiz questions and options (for option text in plots).

class_factory.quiz_maker.quiz_viz.create_question_figure(df: DataFrame, question_text: str, quiz_df: DataFrame = None) → Figure[source]

Create a Plotly bar chart for a specific quiz question, showing answer distribution and correctness.

Parameters:

  • df (pd.DataFrame) – DataFrame containing quiz responses.

  • question_text (str) – The question text to plot.

  • quiz_df (pd.DataFrame, optional) – DataFrame containing the quiz questions and options (for answer text mapping).

Returns:

Plotly figure object visualizing answer distribution for the question.

Return type:

go.Figure

Module contents