I'm an undergraduate at Concordia University, Montreal studying Computation Arts, a research assistant at the Topological Media Lab, and a core member of DATA. I'm interested in responsive media: responsive environments, new computational materials, and hybrid digital/living systems. This is my portfolio for CART360 with Vincent Leclerc. Find my main portfolio here.
RGel (Responsive Gel) is an apparatus capable of synthesizing new forms of quasi-physical "computational matter" in a neutral material substrate. Computational matter is different from ordinary matter in that it is defined only phenomenally – the only properties are affective properties. Consisting of an array of bend sensors and vibrotactile actuators embedded in pliable silicone gel, RGel responds to contortion with rich, realistic or hyperrealistic vibrations that simulate the tactile sensations of arbitrary materials.
[ More ]
"We're always looking for new physics; new behavior that has never been seen before. Once we find it, of course, we start to daydream." – Marc Kastner
RGel (Responsive Gel) is an apparatus capable of synthesizing new forms of quasi-physical "computational matter" in a neutral material substrate. Computational matter is different from ordinary matter in that it is defined only phenomenally – the only properties are affective properties. Consisting of an array of bend sensors and vibrotactile actuators embedded in pliable silicone gel, RGel responds to contortion with rich, realistic or hyperrealistic vibrations that simulate the tactile sensations of arbitrary materials.RGel is conceived not as a device, but a prototype material. As evidenced by databases like Materia, institutions like Material ConneXion[1], and the work of designers such as Linnaea Tillett and Neri Oxman, an industry and a practice focused on generating and using 'new' materials with novel properties is blooming. As further testament, Erica Robles and Mikael Wiberg from New York University and Umeå University will be presenting a paper entitled "Texturing the 'Material Turn' in Interaction Design" at the TEI 10 conference this year. RGel envisions such a "material turn", a future where new materials exhibit more than novel material properties, but controllable phenomenal properties as well. Call it phenomenological material design. As electronic components grow more advanced, smaller, cheaper, and more efficient, the possibility of designing materials with computationally-enabled active or responsive properties approaches reality. RGel invites us to bracket the harsh unchangeable materiality of material itself[2] and think of matter as something programmable, to imagine new class of hybrid materials with responsive capabilities enabled by distributed sensors and actuators. These new active materials could be thought of and used as raw material with which to build responsive interfaces, tools, spaces, and buildings.
For the first prototype of RGel, we established the hypothesis that we could synthesize the sensation of breaking a brittle material such as glass or a cracker. Read more about our initial vision in the proposal.
1. "Every idea has a material solution" reads their slogan
2. As a designer from the future might put it, 'by its current reductive definition'
Technical
The first RGel prototype consists of an 8"x10"x2" slab of silicone gel laced with electronics. It detects contortions with a grid of four fiber optic bend sensors, which are used to drive software sound synthesis. The sounds generated by the synthesizers are then used to control an array of vibrotactile actuators in the gel.
Fiber optic bend sensors were chosen over traditional bend sensors and paper sensors mainly for aesthetic and educational reasons. They were constructed using an IF-E96 LED (emitter) and IF-D91 photodiode (receiver) from Industrial Fiber Optics with 1mm plastic fiber optic cable purchased from Abra Electronics. To create a bend sensor, the jacket was stripped completely and the cladding scraped off on one side of the fiber optic cable to allow a small amount of light to escape. Once this is done, when the cable is bent in the direction away from the removed cladding, more light is allowed to escape, while when the cable is bent toward the removed cladding, more light is retained. This allows for bi-directional sensing, unlike traditional bend sensors. By reading the voltage difference generated by the photodiode in response to the light fluctuation, a measure of the deformation of the fiber optic cable can be acquired. The voltage difference was read by an Arduino, an ATmega-based microcontroller which streamed the data to a laptop over USB.
Max was used to process the sensor data. Because the amount of light leaked while bending away from the scraped cladding is greater than the amount of light retained while bending toward the scraped cladding, an algorithm for measuring and compensating for this effect was devised. In addition to typical smoothing and scaling operations, the signal was rectified to give identical values for bending "up" and "down". Future iterations will take advantage of the bi-directionality of the sensors. The cooked sensor data was then mapped to a function resembling f(x)=(log(x)+2)/2 using the Max "object" [function] to yield a sharp onset.
Synthesis was implemented in Max/MSP. The first presentation of the device used a signal file playback buffer and an instance of IRCAM's Modalys running a plate simulation. Sound from the file playback buffer was used to excite the plate and the output consisted of a mix of both signals. The physical modeling was removed for the second presentation. We plan to use multiple playback buffers and a physical model with an array of spatialized exciters in the future. By placing the exciters at different locations on the plate corresponding to the locations of the bend sensors in the gel, we can create an accurately spatialized multi-channel output signal. Audio from the single playback buffer was split into four channels and filtered appropriately for the destination actuators. Audio sent to the speakers was highpass filtered and DC offset was removed while audio sent to the inertial motors was lowpass filtered. A fifth channel intended for the DC vibration motor consisted of a comparator and a lowpass filter. We found that large pager-type vibration motors respond best to DC pulses.
To retain the DC and sub-sonic frequencies in the output signal, we bypassed the "DC blockers" (highpass filter capacitors) in a Presonus Firebox firewire audio interface. This was done easily by soldering the leads together on the PCB. The DC blockers are the 10 identical capacitors on the top PCB of the device, off to one side and grouped in fours.
Choosing vibrotactile actuators (vibration motors) was the most difficult portion of the project. We conducted in depth research into the available options and acquired as many as we could for testing. Unfortunately due to shipping delays, we were not able to test some actuators before making the mould and we had to make do with what we had. Among the pager motors that we ordered were 12mm and 20mm motors from Precision Microdrives and a generic 20mm motor purchased from All Electronics.
In the prototype, we used two hand-made "inertial" motors and one hand-made "pager" motor (DC motor with eccentric weight). The inertial motors were designed by Hsin-Yun Yao, Vincent Hayward, and Randy E. Ellis at the McGill Haptics Laboratory for the Microtactus project. The parts for the motors were generously given to us by Joseph Malloch from the Input Devices and Music Interaction Laboratory who had manufactured some for use in digital musical instruments. They allow for extremely accurate reproduction of audio signals from the subsonic (<20hz) domain to well into the sonic domain (<1khz). Unlike pager motors, they have excellent transient response which allows them to easily transduce complex signals. The larger pager motor was added to provide "kick" to the signal. By filtering the audio through a comparator, we were able to send short DC pulses that correlated to transients in the signal that provided extra impact. (Researching vibration motors, we found that some had a 'nominal vibration' rating measured in g's. We were not sure what to make of this scale–how much vibration was necessary to achieve what we wanted? After receiving a few rated motors after the prototype was completed, we discovered that even a motor rated at 4g was still insufficient to vibrate the gel in a visceral way (most pager motors are rated at less than 1.5g). Only the larger motor for which we milled a weight had the power we were looking for, but unfortunately we are not able to provide a measure of the vibrational amplitude. What would have worked best would have been large inertial motors. The inertial motors are exceptionally powerful for their size and were they about twice the size, they would combine the best of both worlds: amazingly fast response to changes in voltage and high (hopefully sufficient) vibrational amplitude for their size. Unfortunately we do not have access to the equations governing the proportions of these motors, so without help from the designers, we may not be able to produce larger ones. The options, then, would be to reduce the size of the silicone and aim for a subtler effect, use more inertial motors coupled with large pager motors, or give up on the inertial motors completely.)
In addition to the vibration motors, we used a pair of ordinary speakers (102-1549-ND) to reproduce some of the signal aurally. This was done to experiment with coupling between vibrotactile and aural phenomena. To drive the speakers and the vibration motors, we used a simple 1W, 5V amplifier designed by Mark T. Marshal using the TDA7052 IC. Due to the high current loads required, we used a MOSFET to control the large pager motor. A 5V voltage regulator was used to step down the voltage from a 12V power supply for the amplifiers.
The components were soldered together and set in a rectangular slab of silicone gel. To do this, we constructed a resize-able mould from plexiglass based on advice from the silicone dealer. After trying a number of different silicone products at Sial in Laval, QC, we settled on Smooth-On Dragon Skin 10, a high quality water-clear silicone with a shore hardness of 10A as our choice of silicone.
Originally we had planned to sew the electronics onto a piece of cloth or vinyl, but for the first prototype we experimented with laying the electronics directly into the silicone. This proved difficult as the components floated or, in the case of the bend sensors, insisted on retaining their bent shape. Nevertheless, the final product was functional.
Components
- 4x LED and photodiode mounted in barrel-mounts from Industrial Fiber Optics
- 1mm plastic fiber optic cable
- custom software written with Max/MSP/Jitter
- Presonus Firebox audio interface with DC blockers removed
- 2x custom fabricated "inertial" motors
- 1x generic ~30mm DC motor with custom milled eccentric weight (pager motor)
- 2x 38mm low-profile speakers (102-1549-ND)
- 4x TDA7052-based amplifier circuits
- 4x photodiode amplifier circuits
- 2 lbs. Smooth-On Dragon Skin 10 silicone gel
- plexiglass for display stand and adjustable mould
Conclusion
Though constructing the first RGel prototype, we learned an immense amount about vibrotactile actuators, bend sensing, silicone moulds, analog signal conditioning, and driving motors in atypical ways (i.e. with sound). In constructing the prototype, we made a number of mistakes that rendered the device only semi-functional, but the mistakes have been identified and a second prototype is being planned that will correct the mistakes. In the second prototype, we will: (1) use less silicone and/or stronger motors. The silicone turned out to be too much mass for the vibration motors to actuate sufficiently and we were not able to achieve the desired vibrotactile effects, (2) sew the components onto a piece of cloth before placing them into the silicone to ensure even distribution and the appropriate shape, (3) experiment more with physical synthesis models and a large number of actuators to achieve more convincing spatialization, (4) develop a better signal conditioning circuit for the bend sensors (we were measuring voltage fluctuations of only 10mV from the photodiode). We believe that a new prototype taking these lessons into consideration will achieve our goal of synthesizing the sensation of deforming a brittle object.
References
- Natural Interactive Walking, vibrotactile floor tiles, by Yon Visell and Jeremy Cooperstock at the Shared Reality Lab
- Semblance, art installation experimenting with full body 'just noticeable' vibrotactility by Chris Salter, Marije Baalman, Harry Smoak, etc. (in production)
- TapTap [PDF], wearable that records touches and replays them using vibration motors, by Jeff Lieberman, Oritz Zuckerman, Cati Vaucelle, Leonard Bonanni at the MIT Media Lab and Harvard
- Microtactus, a surgical instrument for amplifying texture, by Hsin-Yun Yao, Vincent Hayward and Randy E. Ellis
Acknowledgements
We would like to thank our mentors and teachers, Vincent Leclerc, Gary Scavone and Sha Xin Wei for immeasurable guidance. Elio Bidinost from the Sensor Lab contributed sage advice and patient help without which the project could not have been realized. We would also like to thank Martin Peach and Avrum Hollinger for advice on constructing fiber optic bend sensors as well as Joseph Malloch and Dr. Marcelo Wanderly for helping us build and contributing the inertial motors.
–Morgan Sutherland & Faiq Hussein
RGel is an apparatus capable of synthesizing new forms of quasi-physical "computational matter" in a neutral substrate. Computational matter is different from ordinary matter in that it is defined only phenomenally – the only properties are affective, felt properties.
[ More ]
"We're always looking for new physics; new behavior that has never been seen before. Once we find it, of course, we start to daydream." – Marc Kastner
I propose to create an apparatus capable of synthesizing new forms of quasi-physical "computational matter" in a neutral substrate. Computational matter is different from ordinary matter in that it is defined only phenomenally – the only properties are affective, felt properties. By bracketing materiality itself, we can begin to think of matter as something programmable. We can imagine new hybrid materials that defy physical laws. These new active materials can then be used as raw material with which to build responsive interfaces, tools, spaces, structures and art objects.
I am interested in creating a prototype 'material synthesizer' using sensate silicone gel embedded with coupled vibrotactile and sonic actuators. The gel, when contorted, will exhibit various vibrotactile and sonic behaviors associated with materials other than gel, be they real or imaginary. My hypothesis is that by exciting the gel strategically in response to interaction, I can synthesize the sensation of manipulating a new material. More specifically, I hypothesize that I can make silicone gel, a flexible, stretchy material, feel brittle.
The apparatus will consist of an array of sensors to sense contortions of the gel, a microcontroller and radio to send the sensor data to a computer, software to extract features from the sensor data and map them to parameters of a physical-modeling synthesizer, another microcontroller and radio to send audio data back to the object, and a custom 'cross-over' to share the audio between an array of vibrating motors and speakers. All circuitry will be sewn onto a layer of translucent fabric and cast into a silicone gel mold.
For sensing, I plan to experiment with fiber-optic bend sensors. I will arrange them in a grid in order to generate a two dimensional contortion map. This will be mapped to a physical model defined in IRCAM's Modalys system. The sound generated by the physical model will drive an array of small 'shaftless' vibration motors as well as small speakers. I may use special transducers designed for vibrating hard surfaces rather than traditional speakers.
Finally, I plan to present the project not as an interface or an art project, but as a demo for a new material about to enter production. I will create a fictional company and set up an engineering-demo-style booth to advertise the project in various high-traffic areas at Concordia and/or McGill University. The project presentation will be partly a performance. I will prepare a routine promoting the near-future (next year) mass adoption of active materials as materials for building and designing interface products.
I'm interested in enchanting everyday spaces. Picture a walk through the forrest. You come across a patch of moss and gingerly step across it, breathing in the fresh smell of amphibious plant matter. What if our cities could feel like this?
Imagine a section of a metro station with floor, bench, and walls made from "responsive gel". Artists could be contracted to script interaction scenarios for these surfaces. The behavior would not be invasive, distracting, or confusing, but calm, passive, and delightful, like the watery squishing of moss. For instance, the floor could be scripted to feel like thin layers of ice that shatter beneath your feet and the bench like a bank of snow as you fall into it.
- Natural Interactive Walking by Yon Visell
- Semblance by Chris Salter, Marije Baalman, Harry Smoak, etc. (in production)
- TapTap [PDF] by Jeff Lieberman, Oritz Zuckerman, Cati Vaucelle, Leonard Bonanni
Resonance is a component of an 'exquisite corpse', a circular chain of microcontrollers that pass data to each other in an endless loop much like the game "telephone".
[ More ]
Resonance is a component of an 'exquisite corpse', a circular chain of microcontrollers that pass data to each other in an endless loop much like the game "telephone". Each microcontroller takes data from its neighbor, transduces it into the physical realm, and reclaims the information with sensors. The modified data is then sent on to the next actor in the network. Resonance uses the data from its neighbor to modulate the pitch of an oscillator. The oscillator drives a sound "exciter", a transducer that is designed to stick to a hard surface and convert it into a soundboard. The exciter is attached to the bottom of a plastic dish, which resonates audibly. Adjacent to the exciter is a contact microphone, which picks up vibrations from the dish. The signal from the contact microphone is used to modulate the pitch of the oscillator. This creates a pitch-feedback loop. As the pitch changes, the character of the pitch modulation changes, thus effecting the pitch and so on. The signal from the contact microphone is then passed on to the next microcontroller in the loop. In addition to the input data, the oscillator's pitch can be controlled in a more coarse fashion with a potentiometer. Resonance is a performance instrument. The plastic dish is filled with water, allowing a performer to modulate the natural resonant characteristics of the dish by tipping it. This both changes the emitted sound and influences the pitch-feedback loop. By tipping the dish and controlling the coarse pitch of the oscillator, a performer can create interesting sounds. Resonance distinguishes itself from typical synthesizers in that the response to parameter changes is not linear. The performer is at odds with the pitch-feedback loop, which is always on the verge of exploding. This puts the performer in a precarious, even perilous position. He must work to keep the system under control while simultaneously attempting to express himself.
Resonance from Morgan Sutherland on Vimeo.
Floracide is a high-risk single-player game in which a player is responsible for the well-being of a live kéfir yogurt culture.
[ More ]
Floracide is a high-risk single-player game in which a player is responsible for the well-being of a live kéfir yogurt culture. To succeed, the player must catch drops of tap water in her mouth, which drip from a balcony three stories above her head. If she fails to catch the drops, she will offset the temperature of the culture, causing it to slowly die. Floracide is a game about balance and empathy, but also a critique of public policy. Tap water in Montreal, as in many cities, is chlorinated in order to prevent the transmission of harmful bacteria. However, in addition to helping to prevent the spread of disease in humans, chlorinated water is toxic to a range or organisms including gut flora, probiotic bacteria in the human digestive system. Floracide forces the player to choose between two evils: either she can drink the chlorinated, saving the culture at the expense of her own flora, or selfishly let the culture die. Floracide was installed between the fifth, sixth and seventh floors of the Concordia University EV building. The three floors share a common area on the fifth floor with balconies on the sixth and seventh overlooking the space. We installed a computer-controlled water dripper on the seventh floor balcony which dripped two streams of drips to the fifth floor. The dripper was implemented as two latex hoses there were pinched independently by solenoids to selectively let water through. The player was instructed to stand between two drip-sensors on the fifth floor, one in front, and one in back of them, which would register missed drops. They were instructed to keep their feet planted and to lean forward or back in order to catch the drips. The drip sensors were implemented as drum heads with contact microphones attached. Registered drips were sent as events to another computer which incremented or decremented an arbitrary temperature value. This temperature value was used to control a peltier element, which was sandwiched beneath a beaker containing the bacterial culture. As the temperature parameter rose, the peltier element heated up, causing the temperature of the culture to rise and vice versa. Yogurt cultures are sensitive to temperature changes, so by changing the temperature of the culture, we risked killing it. Additionally, to give the player a sense of the state of the system, we projected a representation of the temperature and state of the culture onto the bacterial culture itself. A particle system written in Jitter was used to signify the bacteria and the temperature was modulated between blue for cold and red for hot to signify the temperature. Find alternative documentation at the official documentation page.
Floracide from Morgan Sutherland on Vimeo.
Electronic ink is a display technology that allows for non-illuminated electronically controlled displays that could potentially be as flexible as paper.
[ More ]
Electronic ink is a display technology that allows for non-illuminated electronically controlled displays that could potentially be as flexible as paper. It consists of a grid of pixel-sized containers of white and black ink. A grid of electronics underneath the ink layer can charge the containers individually causing the black and white inks to switch places in turn causing them to either appear black or white. While this is a display technology and not a material as such, I believe that e-ink is a first step toward eroding the boundary between the two concepts just as the iPhone (or perhaps the BlackBerry) was a first step toward eroding the distinction between the phone and the computer (though perhaps rather than the distinction eroding, both terms are dissolving into each other and into surrounding categories). Electronic ink is a first step toward imagining displays that are part of clothing, furniture, and skin. They point to a future where displays will not be rigid 'devices', but just that: materials. One can look even further to a time when displays can be sprayed onto a surface from aerosol cans. Imagine a fabric embedded with e-ink like capsules and electronics. Clothing and other textiles could be made to change their patterns on command. Fashion would then not be limited solely to static patterns, but dynamic ones. The dynamic content, as it were, could evolve on different time-scales for different effects: from human-scale where the patterns would appear to shift or flash to longer-than-human time scales where the clothing would change imperceptibly when watched. The patterns could also respond to stimuli from sensors or networked data. Imagine a garment that reflects the colors of the environment, or displays colors complimentary to the environmental colors. For the immediate future, electronic ink promises to enable the creation of displays that rival printed displays in affective quality while offering the capabilities of computer controlled displays: dynamic content and interactivity. We are currently in a time when printed media is beyond its heyday. It is bulky, expensive, wasteful, and limited. What keeps paper alive is, quite simply, its superiority over display technologies as a device for reading text. Paper can be folded, torn and taped together, touched without leaving marks, written on, navigated intuitively, it is cheap and, perhaps most importantly, without a back-light, it is more enjoyable to read. If an electronic display could embody most, or even many of these characteristics, paper would become almost unnecessary.
Vibrobox is a device that converts audio signals into felt vibrations.
[ More ]
Vibrobox is a device that converts audio signals into felt vibrations. Essentially, the device consists of a DC motor with an off-axis cam inside a metal box that takes an audio signal as an input. A rectifier circuit is used to convert the AC audio signal into pulses that the DC motor can use. I personally (I say this because I worked with others on the project and I can't speak for their vision) see vibrobox as a step toward a greater project. Generally, I am looking to affectively synthesize materials, to create an active object that can take on the affective properties of different materials. My idea is to leverage some of the properties of an existing material and to augment them rather than attempting to synthesize a material out of thin air. By creating a vibrating actuator that takes audio as input, a wide variety of complex vibrotactile sensations could be synthesized. So, technically, the challenge is to create an actuator that can faithfully do this, that can convert complex audio signals into complex vibrations. Simple vibrations can be achieved easily with a DC motor with a cam affixed off-axis to the shaft as can be found in so-called "pager motors" found commonly in cell-phones and video-game controllers. These motors, however are designed to be used in a discrete manner, not with complex control by audio signals. The challenge is design a circuit which can take an audio signal, rectify and amplify it into something that can drive a DC motor, find a motor that is nimble enough to respond to split-second voltage modulations and that is powerful enough to create vibrations detectable by humans.
Popswitch is a water-based switch embedded in a popsicle.
[ More ]
Popswitch is a water-based switch embedded in a popsicle. When the popsicle melts past two exposed wires, a layer of water connects them, providing a path of lower resistance than the ice. Popswitch presents a whimsical interaction scenario. A popswitch could be prepared and given to a participant of some event of which an interactive system is part. The participant would be aware that the 'popsicle' was somehow special as they would see wires coming out of the bottom, but they probably would not guess that it was a switch and they certainly would not know what it would trigger. The participant could then be allowed to enjoy the popsicle, perhaps with a tinge of suspense, for a minute or so. At some point, the switch would be tripped by water connecting the contacts and some sort of even could be triggered. An array of LEDs in the popsicle could make a pattern, a chemical reaction could be triggered inside of the popsicle so that it would explode, melt more quickly or ooze sweet nectar, some sort of external event could be triggered like a synchretic explosion or something anti-climactic like the lights turning out. I think the more interesting idea would be to have some sort of actuation embedded within the popsicle, though the popsicle might need to be made unrealistically large to contain complex circuitry. The idea of triggering a chemical reaction reminds me of the bombardier beetle, which, when threatened, combines two reactant chemicals in the tip of its abdomen that instantly explode, deterring predators. The popswitch could be used to create a 'bombadier popsicle', a tasty treat that explodes or shoots jets of unpleasant fluid at you in an attempt to save it's own life. Sadly, popsicles are unable to survive unaided in popsicle-eating weather, so the attempt would be in vain.
Is Water More Conductive Than Ice? from Morgan Sutherland on Vimeo.
LSR is liquid silicone rubber.
[ More ]
LSR is liquid silicone rubber. It can be molded into any shape. What it excites me about it is the possibility of embedding electronics in silicone to create 'active materials' that can be animated with actuators.