On the left of the dataset collection tool, the live video stream is displayed. On the right, an example of the sign is given for the person to perform for collection.
Two signs for the words "beer" and "brown" are shown above using static representations of the DVS data concatenated over 90 samples. As we can see, without the temporal dimension, these two signs are very similar. Shown below are the two signs being performed over time. Here we can see the temporal differences between the two signs.
Our demo will take this spatio-temporal data and use a spiking neural network (SNN) to recognize and differentiate signs which are similar in their spacial dimensions, but different in their temporal dimension.
James (Blake) Seekings (Presenter)
Arshia Eslami (Presenter)
Peyton Chandarana
M. Mohammadi, P. Chandarana, J. Seekings, S. Hendrix and R. Zand, "Static hand gesture recognition for American sign language using neuromorphic hardware," IOPScience: Neuromorphic Computing and Engineering, 2022, doi: 10.1088/2634-4386/ac94f3.
This work is supported by the National Science Foundation (NSF) under grant number 2340249.