Tagged: Mixed Reality

SmartGlass and Selective Transparency

2015 SVVR – The Future of Wearable Displays and Inputs

This is David Holz, co-founder of Leap Motion. Watch this lecture or even just browse over this lecture. There are some very interesting ideas of the future of consumer technology. He gets quite detailed about the evolution of Silicon chips and image sensors as we approach the 2020s. Even mentioning the potential of Smart-Dust….

At first he discusses the impact of the HMD (Head Mounted Display) in the coming years. There are some important leaps that must be made for this tech to truly act in the way we wish. As with all technology at the moment, the real crux of the issue is the power supply! Our advance in ability has far outreached the evolution of energy development and storage. Detaching the headset from the computer will be a crucial step. For the coming years, this could mean that its reliant on the Smartphones battery and advances.

As with any predictions, Holz’s are based on current trends and advances, and will obviously change BUT what he’s saying is very easy to believe.

In the lecture, the most interesting thing for me, and my research, was this idea of selective transparency. I recently enquired about buying an A1 sheet of electrochromic glass (Smartglass). I knew this would be expensive and probably impossible, but I thought it would be an interesting addition to the painting I hope to make. If the sensors in the gallery tripped both the fade of a backlight and the fade from frosted to transparent glass, it could be very interesting. Unfortunately an A1 sheet of this glass was going to be just over £1000….. so its on the back burner for the time being… but I’m very interested post MA. On this subject, I’ve found an interesting hack to mimic this effect, (in a way) with sellotape and frosted perspex. see-through sellotape on either side of the perspex takes away the frost and creates transparency.

Anyway…. the Selective Transparency Holz is talking about is in the HMD. He explains that between 2017 – 2022 we could be using headsets that could switch from virtual to mixed / augmented reality at the touch of a button (or 2020s equivalent of a button, maybe a holo-button?)

Very exciting stuff.

Back to present day, Leap Motion have also made some advances as is shown in the Video (though I’ve added some GIFs below). As the LM uses two infrared cameras, they have begun taking the camera feed of your hands and using it instead of 3D modelled hands. Being able to recognise your own hands in a digital landscape will give a strange sense of realism to the experience. Holz explains that this pushes beyond the idea of the “un-canny valley”.




Here are a few more GIFs showing the possible interaction tech we could have with our computer softwares. This is very exciting, finally the mouse and keyboard won’t be the natural interface to a computer, and instead we have……. our hands…….


This is clearly brilliant for art creation.


Here’s a load more to celebrate all this excitement.





NO_THING- by Innovation Lab of Milla & Partner GmbH

This ‘smart space’ helps look at a world without devices. The concept of the work was to create an interactive, augmented experience without a device-based equipment. The environment senses a plastic sheet and projects personalised messages onto it whilst you move around the space. The image on the wall has a variety of games that allow you to interact with the technology. I can only imagine that its very freeing to have this interaction with technology, without holding a device. Probably as freeing as the Nintendo Wii’s first release.

Considering my interest in VR, AR and now MR (Mixed Reality), its good to see concepts around digital-physical augmented space without the use of devices. Perhaps this is a better progression from anti-social personal computing to open source sensor based environments. Or maybe a mix between the two? Who knows!

Photos, Articles and Video link below……

VIDEO:                  http://www.creativeapplications.net/openframeworks/no_thing/

NO_THING – An infrared light framework that turns (almost) anything into a device

“We get to do a lot of thinking about the future of interaction,” says Milla & Partner’s Ingo Wörner, who together with Fabian Fuchs spearheaded NO_THING’s in-house development. Having realized award-winning exhibitions, show rooms, and trade shows, their team knows the limitations of the current generation of smart spaces all too well. “We were frustrated with solutions available to us at the time. While the arrival of audio guides, VR, AR or tablets has dramatically improved how contextual information can be accessed within a space, they create largely solitary, device-driven experiences; devices that have to be lugged around and that distract as much as they reveal,” says Ingo. Another problem is technology itself: devices age quickly (particularly when they go through many hands), require maintenance, and updates. With NO_THING, Ingo and Fabian tried to get rid of the technology altogether (they describe the experience as “close to NO_TECH”) by taking full control of the environment. “We wanted to create a smart space that is aware of my presence and that provides relevant information when and where I need it, without me having to carry around any equipment, punch in numbers or navigate a screen. To achieve that we use nothing but light — visible and invisible.”

To project onto a moving target, which is what NO_THING does at its core, the team first built prototypes with Kinect systems, openFrameworks, ARToolKit and OpenCV. To increase tracking precision, stability and overall performance across different light settings, NO_THING now uses IR (infrared) light, OptiTrack motion capture cameras (with in-camera blob tracking) and IR-reflecting foil for the markers. The tracking data can be accessed via a Node.js-based andSocket.IO-connected tracking server and allows a wide range of interactions – from tilting, rotation, and gestures, to movements and accessing pre-defined hotspots. The 3d projection mapping onto the tracked cardboard ‘devices’ is done in Pandoras Box (a commercial media and show control system) using a webkit layer as texture. The advantage of the latter: all content and interaction can be built and delivered with standard web technology.

NO_THING An interactive low-tech device Single user interaction_4

A ‘holodeck’ degree of freedom demands a lot of the tracking and projection. NO_THING currently performs at an impressive 60 frames per second (a latency of 3 to 4 frames is largely intentional to ensure a stable image) and with a ‘device’ angle of up to 80 degrees. Additional sensors on the device would increase the tracking precision even further.

“Given the trend towards miniaturization and mobility of projection technology and seamless connectivity there’s an infinite potential for meaningful mixed-reality applications,” says Fabian about NO_THING’s future. From deploying the next generation display and lighting technology (miniature LEDs, lasers) to interfacing with ‘always on’ devices to the prospects of taking NO_THING outside of “controlled environments” and to the street — “It is as if the future has met us halfway.”

For now, the project’s biggest feat remains the manifestation of the interface. “The surprise transformation of a simple piece of cardboard without any electronics into an interactive device is magical. It gets me every time and personally, it’s what I appreciate the most,” says Fabian. “And while the commercial applications, as evidenced by our Expo 2015 use case, are myriad, I’d really like to see an artistic take on it, maybe as part of an installation or performance.” So do we.

NO_THING’s Expo Milano 2015 setup was developed together with Jan Hüwel of coolux (producers of Pandoras Box).


NO_THING An interactive low-tech device Technical-setup_2

An interactive low-tech device

16092015 – Back to School Shoeshine


My research has focussed on two elements. Digital Beings….. and the Virtual Space they inhabit….. The way I have attempted to represent these ideas has been highly experimental, and the hearty balance of media I wished for at the beginning of the course has yet to reach fruition. The act of researching the alternative to a real being in physical space from the point of view of a physical person naturally promotes a level of balance between the two but practically, in some cases, I’ve found more comfort in digital process over physical. My desire to physically paint has hopelessly diminished over the past 6 months, and instead an interest in combining physical drawings with 3D animation and processing has evolved.

1395702524740  47733c30aaf3cdfaf25f91a696d093ab 

A mixed media approach has always been my direction, however, their integration with one another is crucial. I feel my successes in the previous year have been in one medium, never the integration of many.  For both the mini workshop and the Interim show, I’ve produced physical canvases, one oils, one inks (and wool)… the concepts behind these works have differed, “Society of Spectacle” visualised the ‘meta-city’ and its disconnected inhabitants, constantly missing the spectacles in front of their eyes whilst focussing on endless handheld distractions. “Congregation” is a composition of red-blooded empty shells of people each captivated by their own individualised sermon being read to the them by their personal devices. These works have focused on information addiction, which maybe goes hand in hand when trying to decipher the definition of Digital Identity, but then again, each individual represents identity differently, and its definitely a horribly broad umbrella term. After spending most of first year looking at the definition of digital identity in a very singular way, towards the end, I began to realise my interest in the representation of religion. I’m not a religious person, but I’m fascinated by religious art. In my eyes, nothing more important could have happened to humanity than the invention of religion. The ethics, arts and evolutions that have come from it are unrivalled. There are of course some very nasty moments in history where religion can be blamed, but I’m looking at it positively. Many of the worlds treasures would not have been created without a belief in God, and in simpler times, the question of the Unknown may have been a much more frightening prospect.



These thoughts about the online individual, his offline counterpart, how these individuals bunch up together to make communities and their eradication of traditions over facts, have then often been placed in images attempting to visualise the complexity of internet traffic?… The results have been aesthetically complex but conceptually very simplistic. I’ve spread my focus across a number of keen interests, and need to find more depth. Maybe thats the definition of quantitive over qualitative.  But do these ideas not all come hand in hand? If we are to look at the stained glass windows of church interiors, the likelihood is the vast glass areas display what appear to be very complex aesthetics, but the stories behind them are intentionally made to be accessible to the masses. Or in fact maybe its the importance of an ethical question made simple to connect with the masses. Either way, trying to present overly complex ideas only promotes exclusivity, which has been, in my mind, the worst aspect of the contemporary art world in the past century. Then again, overly inclusive themes and concepts can be deemed ‘pop’ and sellable in an increasingly financial art world. As with anything, either side of the spectrum is extreme. I read an interesting article by Guardian art critic Jonathan Jones on Ikeda’s ‘Supersymmetry’ and how he believes that although the work was an impressive spectacle, the perceived complexity of it only strengthened Jones’s belief that Ikeda had no idea what was going on in CERN. Particle physics is complex, and therefore Ikeda represented it with aesthetic complexity…. but unlike Ikeda’s work… the complexity of CERN’s research actually means something. This has brought me back to my own work, and visualising internet traffic… Obviously the internet is a busy place, but empty lines can’t represent its manic behaviour, whats its made up of? what times and when is it used? The lines must be textured with media, or better yet, represent actual data.




Networks-ICT-digital_Thinkstock_135810412 lores carousel shape

My work with reactive sensors came to a halt over my sheer lack of experience in the area, yet had I successfully completed my aims with the project, I would not have considered the conceptual significance behind the ability to “turn on” a painting with your presence. At first. I felt the crucial aspect was to reflect the necessity for the individual to make an active choice to interact with their tech, and the internet. Now, its about why they want to. There was an interesting sets of stats released in the news about a week ago claiming that collectively people in the UK use their smartphones 1.1 billion times a day…. the average being around 40/50, and some extremists as high as 1000 times a day. What used to be an extension of communication has become an integral part to the way we think about everyday actions. But what are these interactions? Emails? News? Social Media? etc etc… Another approach is the Angler fish, and its use of light to attract its prey. This may be a route I take a little further. Going forward with the light – distance project, I’d love to create physical versions, however for the time being, and the recent success I’ve had with Leap Motion, I’m going to create it digitally. The narratives of these works will be an intersection of anatomy and architecture, fact over faith, the growth of global communications and the rise of narcissism in the online world.




Digital Architecture – Another vague term I’ve been using… At the start this was to me the information that makes up the web, and the forms they create when moving. This is still the case, however, I want my work to be more accommodating for people, rather than just residing amongst floating bits of spaghetti….


Its the beginning of 2nd Year, and quite worryingly, I’m beginning to think about my final piece. There a elements of it appearing in some of the work I’ve produced, but the equilibrium is yet to be established both in concept and process. I’m excited to get going, if only I could get into the building! Oh well… More self-obsessed dribble to follow.