Abto Software
Sep 27, 2023
No image
Completed
CV enabled American Sign Language recognition
$10,000+
2-3 months
United States
2-5
Service categories
Service Lines
Artificial Intelligence
Mobile Development
Web Development
Domain focus
Education
Healthcare
Programming language
Python
Frameworks
TensorFlow
Subcategories
Artificial Intelligence
Deep Learning
Challenge
Abto Software took part in an ambitious competition to design a model to recognize and classify ASL signs, aimed at education gamification.
Having covered initial discovery, approach determination and evaluation, our team created a TensorFlow Lite language model, trained on data extracted using the MediaPipe Solution.
To allow the algorithm to run across devices and limit the latency, the videos aren’t stored on a public cloud. That means the inference must be smoothly conducted on the user’s device.
Abto Software took part in an ambitious competition to design a model to recognize and classify ASL signs, aimed at education gamification.
Having covered initial discovery, approach determination and evaluation, our team created a TensorFlow Lite language model, trained on data extracted using the MediaPipe Solution.
To allow the algorithm to run across devices and limit the latency, the videos aren’t stored on a public cloud. That means the inference must be smoothly conducted on the user’s device.
Solution
The project's main goals:
1. To develop an ASL recognition model prioritizing accuracy
2. To ensure the machine learning model can run on Android and iOS
The stages our team has covered:
1. Preliminary discovery – we explored different approaches to gain more insight into the best practices
2. Architecture design – we designed a model to associate data with the corresponding ASL signs
3. Model training – feeding the built model with the training dataset to help it learn different patterns
4. Model evaluation – providing the created model with the testing dataset to compare the predicted output with the actual labels
5. Model fine-tuning and optimization (iterative tweaking and retraining to achieve better accuracy)
6. Model conversion into the requested format (TensorFlow Lite)
The project's main goals:
1. To develop an ASL recognition model prioritizing accuracy
2. To ensure the machine learning model can run on Android and iOS
The stages our team has covered:
1. Preliminary discovery – we explored different approaches to gain more insight into the best practices
2. Architecture design – we designed a model to associate data with the corresponding ASL signs
3. Model training – feeding the built model with the training dataset to help it learn different patterns
4. Model evaluation – providing the created model with the testing dataset to compare the predicted output with the actual labels
5. Model fine-tuning and optimization (iterative tweaking and retraining to achieve better accuracy)
6. Model conversion into the requested format (TensorFlow Lite)
Results
The designed ASL model is optimized for deficient internet connection, which makes it suitable for use even in developing countries.
The described ASL model doesn’t use original input (user video’s), which preserves patient security and privacy and ensures regulatory compliance.
With integrated ASL recognition into products and services, people with hearing loss can enjoy:
- Improved communication
- Educational support and gamification
- Social inclusion
- Practical implementations (convenient control over technology, including phones, wearable devices, and home automation systems)
The designed ASL model is optimized for deficient internet connection, which makes it suitable for use even in developing countries.
The described ASL model doesn’t use original input (user video’s), which preserves patient security and privacy and ensures regulatory compliance.
With integrated ASL recognition into products and services, people with hearing loss can enjoy:
- Improved communication
- Educational support and gamification
- Social inclusion
- Practical implementations (convenient control over technology, including phones, wearable devices, and home automation systems)