I developed the interactive software code for Luciferins, quite some time ago…. last April in fact. I’m using Max/MSP, a visual programing language for artists, educators, musicians and researchers, where you can do a number of things with sound, image, physical computing and other media. Using the openCV library, coupled with the jit.freenect object and a depth camera (the Kinect 1414 in my case), you can see in the dark! This enables me to sense people without a specific light set-up.  I’m running some basic blob tracking and extracting the coordinates of the blob. So much fun! I had that rocking and rolling by the time I wrapped it up to focus on other (burning) areas of the project (read: fibers, graphics, and sound).

The graphics are being developed in Processing. As I was developing the graphic animations, I realized that for the portals to work properly, I need a blob ID for each blob, in addition to the x/y location. This blob ID allows for a blob to be identified and distinct from another blob. This is helpful so as we need to know how long a blob has been in one place, especially in places where blobs might cross paths. This was a pretty straight-forward tweak to make in my interactive code. Back into Max we go!

By this point, it’s around October. I hook the kinect up and run my Max patches, then run my Processing sketches that sync up with the Max patch. I start tweaking the kinect parameters for the setup, aaaaannnndddd Max has a drop in communication with the Kinect…. something like “Could not process events” comes up in the Max window. Hrrmm. I go to re-trigger the kinect, and the entire Max application crashes. I start Max back up, load and run my patches, and the same thing happens over and over again. Ugh.

I had to ruminate as to why this would be happening. Internet searches weren’t procuring fitting leads. I worked to isolate the issue and to note related issues. 

After running tests at Harvestworks, I was seeking up update my OS. Inadvertently, I updated to the most recent Mac OS at the time – Catalina. It caused many other headaches with other applications as Catalina was on the bleeding edge. Other applications hadn’t incorporated support yet to operate on such an architecture. Headaches abound, as the only way to rewind to an earlier OS is through Time Machine. I don’t use Time Machine as it seems to be more of a space hog than it’s been worth. Until now. (Note to self: 1 Time Machine backup a year at least; other smaller calculated backups much more frequently). So, here we are unable to rewind to another OS and the current OS makes nearly everything else inoperable. I saw no other option, but to go nuclear.

Nuclear, as in to wipe the drive completely and install a new OS. This can get tricky. I called in a dear friend from work, Elton to help me out. After many hours, he was able to install Mojave. This is a more recent version than Yosemite (my former operating system), but not as recent as Catalina. This was also the same operating system as successfully tested at Harvestworks. I am grateful to Elton for his time and efforts. 

Now with a new operating system, I fired up my Max patch with the Kinect connected and……(gasp)…….(gasp)…….it crashed again. 

Same issues. Max complains about lost processes and crashes. 

Grrrr…. What is going on? Where could this issue be?

The issue is seeming to be deeper and more specific (as in local to my machine). 

Since this is my principal machine, I’m not inclined to take this apart and to trouble shoot. I can’t afford it to not go correctly. As a result, I dropped it off at Apple. It was due for a new battery, as well.

After 3 days, I was able to pick it up from the Apple store. At this point I tested the machine with my code/application, and the kinect. After plugging things in……the kinect was still dropped and Max crashed after re-initiation of the patch. Hmmmm…something is not right.

I called one of my best mates, Alex to help me brain storm what could possibly be going on. He came over to my studio and hung out for a solid day. He put my computer to work!

Investigating the issue……

After downloading a program to check the USB communication, LSUSB, we ran a simple openCV script to test the USB connection between the Kinect and the computer, especially while it was in use.

Sure enough…. we could see that the USB communication to peripheral devices were getting dropped, especially when other applications were mildly taxing the CPU.

Checking the output of the program in a terminal window, we could see the devices go offline and come back online, as CPU waxed and wained. This lead us to believe it was some sort of hardware failure.

I took the laptop back to Apple. After speaking to a manager, technician (twice), an Apple associate (or 3), dropping my laptop off for 5 days, and speaking to another Apple manager….I had my laptop back with a new logic board, I/O board and cables.

Considering the 90 day warranty on service and parts, I was eager to test this turkey, and see what happened…..and if this sorted my issue with the USB connection.

I hooked up the USB, ran the LSUSB app, as well as my code and everything seemed to be humming along just fine……

Until it wasn’t.

After running my code a several minutes of running, the computer blue screened:

Something is definitely wrong.

I powered the computer down and disconnected the kinect. After rebooting, I tried a different Kinect and so far, things seem to be running okay. I did one test with the project running for 6 hours, and there weren’t any drops, or hitches.

I have been testing with various kinect 1414 cameras. I intend to keep running tests, as I know this problem will inevitably surface again. Perhaps something happened with the previous camera. I’ll be trying different camera setups, as well as watching the CPU power while the project runs. It’s critical to narrow this down.

Regardless, the drum beat is getting louder. What drum beat, you might ask? The drum beat to upgrade my technology. The kinect 1414 is an old depth camera. As technology keep evolving, eventually the software that is built won’t be able to communicate with older hardware and software. There are newer depth cameras on the market. I’ve been exploring an Orbbec Astra. Initially they had a development environment for a Mac OS. It seems in recent years, they’ve yanked it, as it’s no longer supported. It is supported on Windows, Linux and Android. It seems that to work with a new camera, I’m also going to need a different operating system. I could run a parallel operating system, but that gets a bit tricky when the virtual machine needs to communicate with the hardware (ports, ie, peripheral devices like the kinect). So change is coming. It’s just a matter of when. Inevitably the project is going to need it, as it needs a stable environment to operate properly. I’m hoping to stick with a Mac OS. Perhaps another depth camera will come on the market. Or maybe I need to get a windows machine. 

In the meantime, I’m going to be running continual tests. I’m eager to get the interaction code (Max) synced with my Processing code (graphics). I’ve had some really exciting developments with the graphics. I’m also eager to see these graphics working with the fibers. There is lots to do with this project. Here goes!