With Luciferins, I’ll be using a depth camera in two ways: one being to sense where people are located within the installation; the other being to see the fiber structures to then project the images onto them.

I’m super comfortable using Max/MSP/Jitter and OpenCV. I’ve used it a number of times for pieces and teach the software as part of my Interactive Installation class. I threw together a quick patch using the depth camera to track a user’s hand. The camera senses a “blob” within a certain field range in front of the camera. If a blob is within a designated range, it will track it and place the graphics at that point. Here’s a simple demo.

This is all fun and good, but I need to be able to track multiple blobs within a range, not just one. Also, I need to understand where these blobs are in space, so I can also know which fiber structure I need to place the graphics upon.

I’ve been programming and refining a Max patch, which utilizes blob tracking (thanks to the work of  Jean-Marc Pelletier, Nenad Popov and with help from Andrew Roth for their development of the jit.freenect object!) and built upon it to generate the x, y coordinates of a user in space. And then, using OSC to communicate these locations to Processing, in order to trigger the graphics. 

This was working all fine and good. I put the patch down for a few months, as I’ve been doing some heavy fiber work over the last several months. It’s been a welcome break from sitting at a computer screen.

Technology is a very delicate thing. It’s highly dependent on the architecture (computer chips and physical computer make-up) and the software. As you might already know and understand, technology is constantly moving, changing and evolving. While this is what brings exciting changes to our daily lives, it can also bring 1000 headaches.

Upon leaving this patch dormant for a couple of months, I fired it up again with the Kinect attached. The patch kept crashing allow with the application. This spelled trouble. Yet, where does the trouble lie? Isolating the problem is probably 50-90% of the work to overcome this. I began my voyage. Is the issue with the camera? The patch? My laptop? My OS? My laptop hardware? Oh, there are so many places to go here.

The issue repeated with other Kinect cameras. So, it most likely is not the camera itself, but perhaps is something with the third-party object that integrated with Max. An updated version of Max didn’t rectify anything. So, I pulled out an Orbbec Astra depth camera to see if I could get this camera up and running. Orbbec has been coming along for some time. They used to have a library for OSX development at one time (I believe). But now it’s only for Windows. I suspect that this is because it is built with opeNI (which Apple bought and then shut down) versus the lib.freenect environment which was sponsored by Adafruit and is free and openSourced (thank you, Adafruit). Open source code leads to open development. OpenNI runs on a Windows OS and is much more sophisticated and is easier to work with. Yet, I’m on a Mac OS. I could run Parallels or run Windows off a boot-drive. Yet, that seems incredibly risky since the software needs to talk to the hardware port. I sense this could be tricky with regard to a virtual machine.

I did spy that others got the Orbbec Astra working on an OSX through running a Processing library papART. I set to work to try the same process. This could make things easier, developing the piece largely in Processing, rather than a combination of Max + Processing. I ran into some issues and got a bunch of hairy Processing errors when trying to get the papART library running. It seems that I needed an older version of Java installed on my machine.

Oh technology! It breeds so many…..challenges/opportunities and requires so many decisions. It’s critical to be mindful of time, energy, and resources. And to be constantly re-strategizing. Being nimble is key.

While I could install another version of Java, it doesn’t sound like a good decision to me. It also sounds like I’d need a back-up of my machine and a good chunk of time to do this. The ultimate goal of working with the Orbbec Astra camera and depth data was still contingent on several other factors with the papART library. I decided to shelve the Astra for now.

I also took a break. Breaks are good. They help things cool. Give things space and rest. New ideas are then able to come through.

Another possibility is to reprogram all of the blob tracking in Processing. While this is totally possible, it will also set me back a sizable chunk of time, as I’d have to develop the patches from scratch on top of Greg Borenstein’s library.

I began to think more about the problem and realized I had not isolated it properly. Where was the problem? Understanding the location and having a high amount of confidence would justify a size-able time dump. I needed more information. 

I called up the lovely CarolP at Harvestworks. They have a couple machines down there and different versions of Max. Running my patch on one of their machines with my camera will let me know if the issue is with the camera, my machine, my OS or my version of Max. After booking some time in their studio and running my patch for hours, I’ve now deduced that the issue is not with the patch, nor the version of Max, nor my Kinect. This was great relief. I don’t need to re-program anything. Yet, I do need to sort out the issue of the patch running stably on my machine. Harvestworks was on a newer OS than my machine, yet a similar version of Max.

I proceeded to update my OS X from Yosemite (10.10) to Catalina (10.15). Yes, that’s quite a jump. A big jump. A jump I wish I had not taken. The Apple store does not make it easy to find more recent (but not current) versions of their OS. I could not seem to find 10.11 or 10.13 or even 10.14. I do know that the patch and camera work well on 10.13 and 10.14. But instead, I updated to 10.15 and it’s brought a mushroom cloud of more issues with other applications and operations. Many, many more problems.

I tried to roll back to an earlier version of OS X. It IS possible. Yet, right as I embarked on this process, I got a notice for a (4 gb!?!) update from Apple regarding 10.15. And I clicked it. Oh, self! Why did you do this instead of heading back to the safety of an earlier version?

The patch still didn’t run. Upon trying to roll back from this latest version of 10.15, I was faced with this new roadblock:

My choices now are to either install 10.13 or 10.14 from a boot disk. Or, to use TimeMachine to rewind to an earlier moment on my laptop. The last time I used Time Machine to back up my laptop was a few years ago. (I have my own method for backing things up that takes up far less space than Time Machine.) At this point, it looks like Time Machine is my next option.

It’s been a long voyage. I am still at sea.