ಅಕ್ಷರ ಕಲಿ - Kannada Alphabet Learning Application

ಅಕ್ಷರ ಕಲಿ (Akshara Kali) is an interactive Kannada alphabet learning application designed to help children learn and practice Kannada script through guided tracing and personalized progress tracking. The platform integrates a CNN-based handwriting recognition model to evaluate handwriting accuracy and enhance learning outcomes.

The application is built using Next.js for the frontend, with NextAuth.js for authentication and Tailwind CSS for styling. The backend is powered by FastAPI, with MongoDB managing user progress and authentication data. The handwriting recognition model is developed using TensorFlow/Keras, with OpenCV for image preprocessing.

Demo

Landing Page

OAuth Page

Menu

Practice Module

Learn Module

Progress

Mini Games

Bubble Pop

Snake Game

TicTacToe

Bucket Catch

Jigsaw Puzzle

Features

• Two Learning Modes:

  • Practice Mode (Guided Tracing): Helps learners trace Kannada letters with guided assistance.
  • Write Mode (Freehand Writing): Evaluates handwritten characters using AI-powered recognition and provides corrective feedback.

• Letter Pronunciation & Haptics:

  • Each letter is accompanied by audio pronunciation to help children learn the correct sounds.
  • Haptic feedback enhances engagement by providing tactile responses during tracing exercises.

• Thematic Learning Elements:

  • Animal Illustrations: Letters are paired with animals to make learning more engaging for children.

• Interactive Mini-Games (With Score Tracking):

  • Bucket Catch: Letters fall from the top, and the user must catch the correct ones in a bucket.
  • Tic-Tac-Toe (Kannada Letters): A classic game adapted to reinforce letter recognition.
  • Jigsaw Puzzle: Players piece together Kannada letters to complete the puzzle.
  • Snake Game: The snake must eat only the correct target letter to grow, while incorrect choices reduce points.
  • Bubble Pop: Users must pop bubbles containing the target letter while avoiding incorrect ones.
  • Previous Best Score Storage: The highest score for each game is saved, allowing users to track their progress and try to beat their best performances.

• AI-Powered Evaluation:

  • A Convolutional Neural Network (CNN) analyzes freehand writing and assigns scores based on accuracy.
  • Users are rated out of three stars for each letter attempt.

• Progress Tracking & Adaptive Learning:

  • Low-rated letters are prioritized for additional practice to reinforce learning.

• Secure Authentication:

  • Google OAuth integration allows seamless login while maintaining data privacy.

Deployment Details

• Frontend: Vercel
• Backend: Render
• Database: MongoDB Atlas

Environmental Variables

Create a .env.local file in the frontend directory with the following variables:

MONGODB_URI=your_mongodb_uri            # MongoDB connection string
GOOGLE_CLIENT_ID=your_google_client_id  # OAuth client ID from Google Cloud Console
GOOGLE_CLIENT_SECRET=your_google_client_secret  # OAuth client secret
NEXTAUTH_URL=http://localhost:3000      # Your application URL
NEXTAUTH_SECRET=random_string           # Random string for NextAuth session
NEXT_PUBLIC_BACKEND_URL=fastapi_address # URL of the FastAPI Backend Server
DATABASE_NAME=database_name             # Database name

Create a .env file in the backtend directory with the following variables:

MONGODB_URI=your_mongodb_uri            # MongoDB connection string
DATABASE_NAME=database_name             # Database name

Getting Started

Running the Frontend

cd frontend
npm install --legacy-peer-deps
npm run dev

Running the FastAPI Backend

cd backend
python -m venv env
source env/bin/activate
pip install -r requirements.txt
uvicorn api.main:app --reload

Kannada Handwritten Alphabet Recognition Model

Dataset and Preprocessing

• Dataset:
  The dataset can be obtained from IEEE Dataport.

• Image Processing:

  • Images are converted to grayscale and resized to 28×28 pixels.
  • The dataset is split into training (80%) and validation (20%) sets.
  • Labels are one-hot encoded using tf.keras.utils.image_dataset_from_directory.

• Data Augmentation:

  • Applied to the training set to improve generalization.
  • Techniques include random rotation, zoom, and translation using Keras' Sequential model.

CNN Architecture

The model consists of three convolutional blocks:

Block 1:

• Two convolutional layers (32 filters, 3×3 kernel, ReLU activation) with batch normalization.
• Max pooling and dropout (25%).

Block 2:

• Two convolutional layers (64 filters, 3×3 kernel, ReLU activation) with batch normalization.
• Max pooling and dropout (25%).

Block 3:

• Two convolutional layers (128 filters, 3×3 kernel, ReLU activation) with batch normalization.
• Max pooling and dropout (25%).

Final Layers:

• Flattened output is passed through:
  • A dense layer with 128 neurons (ReLU activation), batch normalization, and dropout (50%).
  • A final dense layer with softmax activation to classify 49 Kannada characters.

Model Training

• Compilation:

  • Optimizer: Adam
  • Loss function: Categorical crossentropy

• Callbacks:

  • Early stopping prevents overfitting if validation loss does not improve for five epochs.
  • Learning rate reduction is applied when validation loss plateaus.

• Training Configuration:

  • The model is trained for up to 480 epochs with early stopping enabled.

• Model Saving:

  • The trained model is saved as full_model.h5 for inference.

Model Performance

• Test Accuracy: 98.45%