Project Overview
Elephant Scale, a leader in personalized learning solutions, aimed to develop an end-to end conversational test creation tool. This tool would automatically transcribe audio recordings of online classes, generate quiz questions, and manage backend data pipelines. By leveraging AWS AI services, Elephant Scale sought to enhance personalized learning experiences for students. Avahi, an advanced tier AWS partner, was engaged to implement this innovative solution using AWS services such as Amazon Bedrock, AWS Transcribe, SageMaker, S3, RDS, Lambda, and API Gateway.
Challenges
Elephant Scale faced several challenges in creating an automated test generation tool:
- Real-Time Processing: Capturing and transcribing audio recordings of online classes in near real-time.
- Data Management: Efficiently managing and processing large volumes of educational content and audio data.
- Accuracy: Ensuring high accuracy in transcriptions and quiz question generation.
- Scalability: Building an infrastructure capable of handling multiple concurrent users and large datasets.
- Integration: Seamlessly integrating the new solution with existing educational platforms and data sources.
Solution
Avahi proposed a comprehensive solution to address these challenges, focusing on developing a proof of concept (POC) for the automated test generation tool. The project was executed in four phases: Discovery & Planning, Design & Development, Validation & QA, and Executive Presentation & Handoff.
Key Deliverables
- POC Development: Building the initial POC for an end-to-end automated testing tool on AWS.
- Real-Time Transcription: Developing a system to generate transcripts from audio recordings of online classes in near real-time.
- ETL Processes: Performing ETL on raw data to generate quizzes after each class and store the output text files in Amazon S3.
- Backend Pipeline Management: Creating an end-to-end pipeline to pull data from whitelisted third-party sources.
- Post-Processing Jobs: Running post-processing jobs based on confidence thresholds selected by Elephant Scale.
- Validation: Ensuring accuracy, bias mitigation, and scalability using industry standard tools.
- Qualitative Review: An analysis report based on manual review to identify common issues and patterns.
Solution Architecture
The solution architecture leveraged various AWS services to implement the deliverables and ensure a robust, scalable platform.
Data Acquisition and Storage
- Amazon S3: Used for storing input documents, audio recordings, course content, and processed data. S3’s scalability and durability ensured efficient handling of large volumes of data.
Event-Driven Processing
- AWS Lambda: Deployed to run event-driven functions that process data and manage various stages of the pipeline. Lambda functions were used to fetch documents and class recordings, create embeddings, and update/store outputs.
Machine Learning and AI
- AWS Transcribe: Employed for automatic speech recognition, enabling real-time transcription of audio recordings.
- AWS SageMaker: Utilized for developing, training, and deploying machine learning models that generate quiz questions and manage the transcription process.
- Amazon Bedrock: Used for scalable managed Generative AI and foundational model services, supporting the overall AI infrastructure.
API Management
- Amazon API Gateway: Provided a managed service to create, publish, maintain, and secure APIs. This facilitated seamless communication between the front-end applications and backend services, allowing easy access to transcribed data and generated quizzes.
Monitoring and Logging
- AWS CloudWatch: Implemented for comprehensive monitoring and logging. CloudWatch provided real-time insights into system performance, user interactions, and the health of the deployed models.
Project Impact
Enhanced Personalized Learning
The automated test creation tool significantly improved personalized learning experiences for students by providing real-time transcription of online classes and generating tailored quizzes. This led to increased engagement and better learning outcomes.
Operational Efficiency
By automating the transcription and quiz generation processes, Elephant Scale saw a reduction in manual efforts and operational costs. The efficient data handling and machine learning models ensured timely and accurate delivery of educational content.
Scalability and Flexibility
The architecture designed by Avahi ensured that Elephant Scale’s platform was scalable, capable of handling increased data volumes and user demands. This provided flexibility for future enhancements and growth.
Improved Accuracy
The use of advanced machine learning models in AWS SageMaker and AWS Transcribe enhanced the accuracy of transcriptions and quiz question generation, ensuring high quality educational content.
Detailed Insights and Analytics
The deployment of AWS CloudWatch and other monitoring tools provided Elephant Scale with detailed insights into system performance, transcription accuracy, and user behavior. These analytics were crucial for continuous improvement and optimization of the platform.
Conclusion
The collaboration between Elephant Scale and Avahi led to the successful development of an AI-powered conversational test creation tool. Leveraging AWS services, Elephant Scale enhanced personalized learning experiences, improved operational efficiency, and built a scalable platform ready for future growth. Avahi’s expertise in AWS solutions played a pivotal role in this transformation, demonstrating the power of Generative AI and machine learning in advancing educational technology.