SignSpeak BIM
About
A real-time computer vision application that recognizes custom hand signs through a webcam using MediaPipe and classical machine learning. The project includes a complete ML pipeline from dataset collection to live recognition, and is being developed incrementally with a milestone-driven approach.
What I Did
Designed the full project architecture and computer vision workflow, then built the dataset recording system and landmark preprocessing pipeline, the machine learning training pipeline comparing multiple classifiers, and automatic best-model selection with evaluation and metadata generation, all in a modular, unit-tested Python codebase.
Results
SVM
Best Model
98.59%
Accuracy
98.55%
F1 Score
Highlights
- Real-time hand tracking using MediaPipe
- Custom landmark dataset recorder
- Automatic dataset validation
- Landmark normalization
- Trained and compared Random Forest, SVM and KNN
- Automatic best-model selection
- Live model loading
- Modular project architecture
- Unit-tested training pipeline
Development Timeline
- Webcam capture, real-time hand tracking & two-hand detection
- Dataset recorder, landmark preprocessing & CSV session storage
- ML training pipeline: Random Forest, SVM, KNN + automatic model selection
- Real-time prediction & sentence builder
- ElevenLabs speech synthesis
- UI polish & performance optimization
Technical Details
The pipeline is fully modular: a MediaPipe Hands recognizer streams 21 landmarks per hand in real time, which are normalized for translation and scale invariance before being logged into session-based CSVs by a custom dataset recorder with automatic validation. The training pipeline benchmarks Random Forest, SVM, and KNN classifiers side by side, automatically selecting and persisting the best-performing model via Joblib, currently an SVM at 98.59% accuracy and 98.55% F1. The codebase is unit-tested end to end, from landmark preprocessing through model persistence, so the pipeline stays reliable as new milestones (real-time prediction, sentence building, speech synthesis) are added.