Saturday, January 18, 2014

Cheap Tricks - Interactive Dance Floor Application with Kinect and openFrameworks

I took a short break from a longer term project I am working on to make something for a New Years Eve party dance floor. I didn’t have a lot of time and it shows one of my favorite aspects of working with openFrameworks: you get a lot of stuff for free via the addons available, making it very quick to implement ideas.

You can see a little demo of the project here:

Cheap Tricks from dizzy pete.

It uses a Kinect and OpenCV to get the blobs (in this case people and my Ikea furniture) then turns them into triangles that are audio reactive, as well as adding in some visual effects with no real purpose other than to look cool.

Kinect Input

Reading from the Kinect is relatively straightforward, and more or less lifted straight from Kinect examples that come with oF. It uses the depth information in a grayscale OpenCV image and does some simple thresholding based on depth settings.

This grayscale image is passed to the stock OpenCV blob finder, then the blob information is passed to ofxTriangle to get a set of triangles representing the blob. I initially was using ofxDelaunay but found ofxTriangle just worked a little better for what I was after.


The audio reactive part is done via ofxFFT by julapy (there are a few out there so make sure you have the right one if you’re taking a look). I originally wanted to do beat detection and have the figures pulse in time with the beat, but that was a little too involved given the time constraint, so decided just to stick with the very low end.

I experimented with various numbers of FFT bins but ended up using the default buffer size of 512, bin size 256 and averaging the lowest three bins. This is the variable ‘low’ that is used throughout the code.

Ideally I would have had a proper audio device feeding in from the DJ’s mixer, but as it turned out I was using the default microphone on my MacBook for audio input, which worked much better than I expected.

Based on the current amount of low-end audio detected, each vertex of each triangle is displaced from its center by some amount. Triangle sizes vary; the torso of the detected body has larger triangles than the hand or arm, how much displacement to be applied is a function of the triangle area.

To be precise it is the square root of the triangle’s area, and can be tweaked by a parameter called jitterSize in the gui. I don’t actually displace each vertex, just two out of three. The code to do all third is there but commented out, I just liked the way it looked doing only two.

Once the vertex displacement has been calculated, the triangle is added to a mesh along with texture coordinates, which are used for coloring. But more on that in a second.

The basic triangle form


By this stage the basic audio-reactive triangle people thing I was after was done, but it still looked a bit flat.

Poking around some OpenCV examples, I saw examples of motion detection that looked kinda cool.

First it takes the difference between the current frame and last frame, which gives areas where motion has been detected. This is then scaled up as a function of low-end audio and GUI parameter glowExtra, then added to a buffer image. The buffer image gets decayed over time, so things fade out as the motion fades.

This looked ok, but I wanted it to be a bit smoother so used ofxFboBlur to add in excessive amounts of Gaussian blur to the motion buffer image. I had to change its source a bit just to play a bit nicer with FBO sizes and drawing, which is why it appears in the source folder and not as an addon.

The blurred motion detection background


Coloring was probably the hardest part for me, and even then it was pretty easy.

There are lots of different ways this could be done, but I settled on using a two-color gradient rendered to an FBO, using a mesh with vertex colors. Each frame, the mesh vertex colors are updated by cycling through the Hue component in HSB space, then the updated mesh is rendered to the FBO.

This FBO gets used in two places. First it is blended with the grayscale blurred motion detection background using a multiplicative blend mode, which takes care of coloring in the swishy aura type effect in the background.

Then the FBO texture reference is bound and the mesh containing our displaced triangularized blobs is drawn as a wireframe. This is where having texture coordinates for each vertex in the mesh is important, though fiddling around I noticed I don’t need tex coords if ofBackground() has been called. Who knows.

Regardless, when the wireframe is drawn, it is colored based on the bound FBO texture reference, which takes care of coloring in our triangle people.

The two layers colored and drawn together


So that is just a quick run through of how it works. I used a lot of addons, and all up it took me about 2 days. I started on the evening of the 29th and was finished by the afternoon of the 31st, just in time. Without the oF addons it would’ve taken me well over a week, possibly many weeks.

People at the party really seemed to enjoy it as well, which is always a great feeling.

I used soundflower during development (and oF 0.7.3 eep), but updated it to use oF 0.8, which is why you see ofxSoundStream in the code.

If you don’t have a Kinect it will use the device camera, but it does not work as well. You can use the far threshold setting for the OpenCV threshold in that case.

The source is up here.

Also I have put the source for Particles up here as well

You can find me on twitter here !

Particles 1 from dizzy pete.

Sunday, November 24, 2013

Just for fun: attractors in R

I have a borderline unhealthy obsession with attractors. I thought I got it out of my system, but here we are. For whatever reason, I felt like making some in R.

You can find the R code here. It uses the attractor function to define density in a matrix, which is how often a given point gets hit. Then we plot the log scaled version of that using image().

Be warned, with a lot of points it can be quite slow. If you play around you might want to drop the n value to 10,000 or so.

Some pictures



If you like this sort of thing, I made a realtime interactive dejong attractor, using a kinect, openFrameworks and a lot of GLSL shaders. You can see a clip of it below, the source is also on my github if you are so inclined.

Attractor from dizzy pete on Vimeo.

You can find me on twitter and G+

Book Review: Applied Predictive Modeling by Max Kuhn and Kjell Johnson

This is a gem of a book.

From the introduction:

We intend this work to be a practitioner’s guide to the predictive modeling process and a place where one can come to learn about the approach and to gain intuition about the many commonly used and modern, powerful models.
…it was our goal to be as hands-on as possible, enabling the readers to reproduce the results within reasonable precision as well as being able to naturally extend the predictive modeling approach to their own data.

The book is structured into four main sections. First is General Strategies, which provides an introduction and discusses things like pre-processing and tuning. 

The next two sections cover regression and classification, each with chapters on linear and non-linear methods, as well as tree and rule based methods, with one to two chapters on practical issues such as measuring performance. 

The final section covers feature selection, predictor importance and a discussion around model performance. 


There are a few things I really like:

It is not an academic or mathematical treatise; the emphasis is on practice, discussing the issues that commonly arise and how they can be approached. Plenty of references are provided for those wanting to dig deeper.

Every example has its data set and code available so one can work through the examples as presented. In most cases they are real world datasets and there is great discussion of the real world issues that arise, what should be considered and the various tradeoffs that can be made.

Discussion and code are separate. Aside from the excellent content, this is probably what I appreciate the most. Each chapter presents its content, with charts where appropriate, while the actual walk through of the code and raw output is in a separate section of the chapter. 

This makes it much easier to focus on the material being presented. It is always difficult to present source code along with discussion. This is not a book about programming per-se, it is about using existing tools to make intelligent and reasoned decisions about the task at hand. It makes a lot of sense to have the code presented separately.

Also, as far as I have read, each chart is at most only one page away from the text discussing it. This is a small thing but I feel there has been serious consideration about the presentation of the material and it has been done very well.

It is not a book about caret, the package of author Max Kuhn. To be honest I would be pretty happy even if it were about caret, which certainly does get some use in the code, but it is relatively package agnostic. 

Final thoughts

This is a great book, providing both the trees and the forest so to speak. I am unaware of any other book with similar content, and I wish I had something like this when I was first getting interested in machine learning. 

There are books that are very introductory, books that cover the details of the algorithms, and books that provide rigorous coverage of the theory, but these are not really accessible to those without a serious amount of mathematics. There are a few equations presented where appropriate, but it is certainly not the focus of the book.  

There are no real shortcomings, though if there were ever a second edition, coverage of time series methods and deep learning would be welcome. I appreciate they are both book worthy topics by themselves, and the latter is still very much a moving target. 

In summary: Great content, well written and well presented. This book would be my top recommendation to anyone looking to get started or working with predictive modeling. Well worth checking out. 

You can read more reviews on amazon here: Applied Predictive Modeling

You can find me on twitter and G+

Friday, October 25, 2013

Another Rcpp/RcppArmadillo success story

I finally had an excuse to try out RcppArmadillo and was amazed at just how well it worked out.

The exact details are a bit beyond a blog post, but basically I had a matrix of roughly 80,000x2 double valued samples (call it A) and needed to find where in another, much smaller matrix (call it B) each of the samples was closest too.

There was some structure to the B matrix, which is 500x2, so it wasn't necessary to iterate through all 500, each lookup jumped around a bit and took 8 iterations.

The pseduocode in R looked a bit like this

find_closest <- function(A, B) {
    for(i in 1:8) {
    return index into B that A is closest too

calc_distance <- function(A, B)  apply(A, 1, find_closest, B=B) 

All up it took about 9.5 seconds on my system, and 15 seconds on another.

The first pass I implemented find_closest using RcppArmadillo, and saw a healthy speed up to around 400 ms, which was a big improvement.

Then I realised I might as well do calc_distance/the apply in C++ as it was just a simple vector of integers being returned.

This gave an amazing performance leap, the function now takes around 12 milliseconds all up, down from 9.5 seconds. On another machine it would take 15 seconds, and ended up taking 10 milliseconds.

I was very surprised at this. I haven't have a chance to dig into the details, but I am assuming there is a reasonable amount of overhead passing the data from R to RcppArmadillo. In the case of apply, this would be incurred for every row/column the apply was running find_closest on. By moving the apply to C++, all the data was passed from R to C++ only once, giving the large speedup. Or so I guess.

I tried two versions, one that traversed A & B row wise, and the other by column. The column one generally seemed faster, 12 ms vs 19 ms for rows. According to the Arma docs it stores matrices in column-major order, which might explain that difference.

Would appreciate any comments or pointers to documentation so I can better understand what is going on under the hood there.

The case for data snooping

When we are backtesting automated trading systems, accidental data snooping or look forward errors are an easy mistake to make. The nature of the error in this context is making our predictions using the data we are trying to predict. Typically, it comes from a mistake with our calculations of time offsets somewhere.

However, it can be a useful tool. If we give our system perfect forward knowledge:

1) We establish an upper bound for performance.
2) We can get a quick read if something is worth pursuing further, and
3) It can help highlight other coding errors.

The first two are pretty closely related. If our wonderful model is built using the values it is trying to predict, and still performs no better than random guessing, it’s probably not worth the effort trying to salvage it.

The flip side is when it performs well, that will be as good as it will ever get.

There are two main ways it can help identifying errors. Firstly, if our subsequent testing on non-snooped data provides comparable performance, we probably have another look ahead bug lurking somewhere.

Secondly, things like having amazing accuracy yet still performing poorly is another sign of a bug lurking somewhere.


I wanted to compare SVM models when trained with actual prices vs a series of log returns, using the rolling model code I put up earlier. As a baseline, I also added in a 200 day simple moving average model.

(S) Indicates snooped data

A few things strike me about this.

For the SMA system, peeking ahead by a day only provides a small increase in accuracy. Given the longer-term nature of the 200 day SMA this is probably to be expected.

For the SVM trained systems, the results are somewhat contradictory.

For the look forward models, training on price data had much lower accuracy than the log returns, and the log return model performed much better. Note that both could have achieved 100% accuracy by predicting its first column of training data.

However, when not snooping, the models trained on closing prices did much better than those trained on returns. I’m not 100% sure there isn’t still some bug lurking somewhere, but hey if the code was off it would’ve shown up in the forward tested results no?

Feel free to take a look and have a play around with the code, which is up here.

Monday, September 23, 2013

Building models over rolling time periods

Often I have some idea for a trading system that is of the form “does some particular aspect of the last n periods of data have any predictive use for subsequent periods.”

I generally like to work with nice units of time, such as 4 weeks or 6 months, rather than 30 or 126 days. It probably doesn’t make a meaningful difference in most cases, but it’s nice to have the option.

At first this seemed like something rollapply() from the zoo package could help with, but there are a number of preconditions that need to be met and frankly I find them to be a bit of a pain.

In a nutshell I have not been able to find a nice way for it to apply a function to a rolling subset of time series data nicely aligned to weekly or monthly boundaries.

All is not lost, there is a neat function in xts called endpoints(), which takes a time series and a period (e.g. “weeks”, “months”) and returns the indexes into that time series for the corresponding periods.

Using this information it becomes easy to subset the time set data using normal row subset operations.

The xts package also has period.apply but it runs on non-overlapping intervals, which is close but still not quite what I want.

In the script for this post there are 4 or so functions of note.

The main one is roll_model, which takes a set of data to be subsetted and passed to the model, the size of per model training and test sets and the period size to split things up, which is anything valid for use with endpoints().

A utility function is train_test_split which also uses endpoints() to split a subset of data into 2 sets, one for training the model, one for testing. In practice it needs to be the same period type as you expect to use with roll_model.

The function that actually builds the model and returns some results is run_model(), which calls train_test_split to get the training and test set, builds a model using ksvm in this example, and sees how it goes based on the test set.

Another utility function is called before that, data_prep which builds the main data object to be passed to roll_model. In this example it takes a set of log closes to close returns, sets Y to be the return at time t, X1 the log return at t-1, X2 at t-2 and so on.

The example model is not a particularly useful way of looking at things, which is not surprising given close to close returns are effectively random noise. But perhaps the script itself is useful for other ideas, and if anyone knows better/easier/alternate ways of doing the same thing I would love to hear about them.

The script is available here.

Sunday, September 8, 2013

Snapshot of the global zeitgeist

John le Carré, author of Tinker Tailor Solider Spy (among many others) and Miley Cyrus give a timely reminder that Australia is a small fish in a big pond.