Side Project - Workout Tracker

After seeing the “pushup challenge” videos taking social media by storm, I decided to collaborate and develop something fun and useful for everyone while the gyms are closed and we’re all relying on good old fashioned body weight exercises.

Using a custom, built-from-scratch CNN trained on a dataset that the two of us personally curated and annotated, we have built a tool that will act as your “virtual spotter” and count your pushups as you complete them in front of a webcam. I may decide to continue developing this, a few future ideas include:

• Make this work on a smartphone
• Develop an “alarm clock” app that won’t shut off until a set number of pushups are completed
• Of course, expand to new exercises like burpees and crunches

If you’re interested in the project, the source code can be found here.

Key Concepts: Python (OpenCV), Machine Learning, Computer Vision, Real-Time Image Processing, Custom Models

Side Project - Fingerpainting

Earlier this week, I got the idea for a little side project - an app that lets you “fingerpaint” in real time over a video. So far, I’ve gotten the foundations of an “eraser” function down (shown in the video) which allows the use the palm of your hand to erase your doodles. The actual finger painting portion is next on the list.

I’ve seen a lot of hand gesture recognition models out there, but they have their limitations - an image processing approach to segment out hands is prone to error in different lighting conditions/background noise environments (eg. if there’s a face in the picture that’s the same color as the hand), while RCNN/CNN approaches give either the region that a hand exists in or the gesture on screen, but not the actual boundaries of the hand. A more robust approach is to use a Mask-RCNN in order to detect the each pixel that contains a hand and its gesture, and then go from there. I used the Matterport Mask-RCNN in this demo, trained on a custom dataset that I put together and labeled myself. It’s a bit choppy in real time, but I attribute that partly to my GPU which only has 4gb vram available - Google Colab’s Tesla T4’s have about a 90ms processing time per image whereas I’m getting about 300ms on my hardware. Nonetheless I plan on replacing the Resnet101 backbone with a mode computationally friendly model, such as MobileNet, to allow for fast inference regardless of hardware. The source code can be found here.

Key Concepts: Python (OpenCV), Machine Learning, Computer Vision, Real-Time Image Processing, Mask Region-Based Convolutional Neural Networks, Custom Models

Side Project - Webcam Eye Tracker

A webcam eye tracker, currently a work in progress, which is intended to perform comparably to more expensive hardware/software for a mere fraction of the price. This began as an experimental side project that was to be tested during my graduate research, however at some point other research endeavors took priority. The end goal was to obtain map an individual’s eye-gaze points to (x,y)-coordinates on a plane - ie: the computer screen. In its current state, the program functions as an accurate iris localization tool. The source code can be found here.

Key Concepts: Python (dlib, OpenCV), Machine Learning, Computer Vision, Real-Time Image Processing

Side Project - Analyzing All Los Angeles Parking Violations from 2018

With a bit of free time this weekend, I decided to do some hobby coding in Python, which I’ve presented here as a Jupyter Notebook with an interactive map and some interesting visualizations extracted from the data. The point of this side project is to create a map of LA according to parking tickets received from last year (2018) and to produce some helpful visualizations/gain some insights. I want to start with the full dataset (~9-million x 19, saved as a .csv > 1gb) to gain more experience handling large .csv files that are cumbersome to work with in Python unless processed in batches (especially when you don’t have much RAM available). (Note: as os 2020 I’d have done this in Spark rather than Pandas to take advantage of its distributed computing features and excellent dataframe implementation).

Key Concepts: Python (Numpy/Matplotlib/Pandas/Folium(Mapping Package)), Jupyter Notebooks, Data Science, Big Data

Cognitive Context Detection With Tensorflow

In previous projects, I demonstrated how to implement a Neural Network from start to finish using only home-made functions written in both Matlab and Python. I’ve uploaded a Tensorflow implementation for the sake of comparison (note that this was written pre-Keras, Tensorflow code can now be written much more intuitively using Keras wrappers). I’ve concluded that Tensorflow offers increased productivity in the form of quicker-to-write code and faster training times. The optimization functions that TF offers are great, and allowed me to train my models in a fraction of the time that Matlab or Python+Numpy in conjunction with the GPU acceleration capabilities. Comparing the two files MatlabResults in the Matlab Implementation and TensorflowResults in this repo demonstrate the notable accuracy improvements that Tensorflow offers as well.

Key Concepts: Tensorflow, Python (Numpy/Scipy/Matplotlib/Pandas), Machine Learning, Neural Networks, Data Science