Brazilian Sign Language Recognition

One challenge in Brazilian Sign Language (Libras) recognition is the absence of a robust dataset that allows the validation of different methodologies. For this aim, our team developed two datasets, which we are using with different Machine Learning techniques to help deaf developing communication with deaf people and human-computer interaction based on visual signs. The first dataset contains 10 recorded signs and the second one 20 signs. Read more to joining us.

First dataset: Brazilian Sign Language (Libras) data set with 10 signs for sign language and gesture recognition benchmark: 1) to calm down (acalmar), 2) to accuse (acusar), 3) to annihilate (aniquilar), 4) to love (apaixonado), 5) to gain weight (engordar), 6) happiness (felicidade), 7) slim (magro), 8) lucky (sortudo), 9) surprise (surpresa) and 10) angry (zangado). Each one of these signs was recorded 10 times by 1 signer, totaling a database of 100 samples. The signs were captured using an RGB-D sensor (Microsoft Kinect) and processed by nuiCaptureAnalyze software. This dataset is publicly available at Zenodo (ALMEIDA, et al. 2019a). Some scientific works related to this dataset are Rezende (2016), Rezende et al. (2017) and Guerra et al. (2018).

Second dataset: Brazilian Sign Language (Libras) data set with 20 signs for sign language and gesture recognition benchmark: 1) Acontecer (To happen), 2) Aluno (Student), 3) Amarelo (Yellow), 4) América (America), 5) Aproveitar (To enjoy), 6) Bala (Candy), 7) Banco (Bank), 8) Banheiro (Bathroom), 9) Barulho (Noise), 10) Cinco (Five), 11) Conhecer (To know), 12) Espelho (Mirror), 13) Esquina (Corner), 14) Filho (Son), 15) Maçã (Apple), 16) Medo (Fear), 17) Ruim (Bad), 18) Sapo (Frog), 19) Vacina (Vaccine) and 20) Vontade (Will). Each one of these signs was recorded 5 times by 12 signers, using a Chroma Key background. Among the signers are men and women with basic to advanced knowledge in BSL. Videos are in 1920 x 1080 MP4 files (30 fps). This dataset is also publicly available at Zenodo (ALMEIDA, et al. 2019b). Some scientific works related to this dataset are Mendes (2019) and Castro et al. (2019).

References:

Silvia Grasiella Moreira Almeida, Tamires Martins Rezende, Andreia Chagas Rocha Toffolo, & Cristiano Leite de Castro. (2019). Libras-10 Dataset [Data set]. Zenodo. http://doi.org/10.5281/zenodo.3229958

Almeida, Sílvia G. M., Rezende, Tamires M., Almeida, Gabriela T. B., Toffolo, Andreia C. R, & Guimarães, Frederico G. (2019). Libras-20 Dataset [Data set]. Zenodo. http://doi.org/10.5281/zenodo.2667329

Rezende, Tamires M., Aplicação de Técnicas de Inteligência Computacional para Análise da Expressão Facial em Reconhecimento de Sinais de Libras. 2016. 108f. Masters Dissertation - Universidade Federal de Minas Gerais, Minas Gerais, 2016

Rezende, Tamires M. et al., Análise da Expressão Facial em Reconhecimento de Sinais de Libras. In: VI Simpósio Brasileiro de Automação Inteligente. 2017. p. 465-470.

Guerra, Rubia R. et al. Facial Expression Analysis in Brazilian Sign Language for Sign Recognition. In: Anais do XV Encontro Nacional de Inteligência Artificial e Computacional. SBC, 2018. p. 216-227.

Castro, Giulia Z. et al. Desenvolvimento de uma Base de Dados de Sinais de Libras para Aprendizado de Máquina: Estudo de Caso com CNN 3D. In: In: VI Simpósio Brasileiro de Automação Inteligente. 2019.

Mendes, Moises. Aplicação de Deep Learning no reconhecimento de sinais de Libras: aspectos técnicos e sociais. 2019. 51f. Trabalho de Conclusão de Curso - Universidade Federal de Minas Gerais, Minas Gerais, 2019