Echoes of Us
An Interactive Memory Museum
Echoes of Us is an interactive installation that invites participants to reminisce, reflect, and potentially reconcile through the narrator’s perspective. Utilizing Arduino-powered sensors, LEDs, projectors, and evocative sound design, this physical collection allows a single performer to unlock the narrator’s core memories, vividly unfolding them for the audience to experience and feel.
Group members: JC Zhang, Yudi Cao & Peter Oke
Experience
A single participant assumes the role of the performer, activating the "memories" embedded in five carefully arranged objects: a small box containing baby shoes, a deflated volleyball, a torn painting, a chair, and a vintage rotary telephone.
As the performer interacts with each object, a father-daughter relationship story unfolds. The baby shoes trigger a projected video illustrating the narrator’s “memory” of his daughter’s first steps. The volleyball prompts a brief display of a high school volleyball game as the narrator expresses his pride in his daughter, leading her team to the finals.
The narrative shifts when the performer engages with the ripped painting, a symbol of a heated clash over her dream to pursue a career as an artist. From this moment, the performer’s choices shape one of three endings: a lingering indifference to their past conflict, a poignant solitude steeped in regret, or a hopeful reconciliation that promises to mend their fractured bond.
Technical Tools & Equipment
Hardware
Story Objects: The installation features five carefully selected objects that convey the narrative: a pair of baby shoes in a wooden box, a deflated volleyball, a second-hand painting, a chair, and a vintage rotary phone. Each object is rigged for sensor integration.
Sensors: Four types of sensors are employed to detect performer interaction:
-
Tilt Sensor: Attached to the wooden box to detect when it’s opened to reveal the baby shoes.
-
Vibration Sensor: Embedded in the volleyball to sense when it’s touched or moved.
-
Distance Sensor: Positioned near the painting to detect proximity as the performer approaches.
-
Pressure Sensor: Integrated into the chair’s seat to register when it’s sat on or pressed.
Projectors: Two 4 K short-throw projectors are used: one for the “memories” triggered by the objects, and the other to display the ending, dependent on what choices the performer made at the end.
Speaker: Plays audio narration, foley, and ambient sound effects.
Laptop: Runs TouchDesigner and stores video/audio files.
USB Cable: Connects Arduino to the laptop for power and data transfer.
Software
Arduino IDE: Used to program the Arduino to read sensor inputs (tilt, vibration, distance, pressure) and send serial data to TouchDesigner. It also receives commands (e.g., LED on/off) from TouchDesigner.
TouchDesigner: Receives real-time sensor data from Arduino. It contains the main logic that tracks the sequence of interactions (baby shoes → volleyball → painting → chair). Only if all sensors are triggered in the correct order, it triggers a phone ringing sound, plays a “hello” voice recording and then triggers the video to play at the end.
DaVinci Resolve: Employed for editing and syncing video clips of the “memories” with the pre-recorded narration, foley sounds, and the background ambience that included a nostalgic soundtrack and the sounds of scribbling on paper. Ensured seamless integration of visuals and audio for a polished experience.
Adobe Podcast: Utilized for sound editing and enhancement, refining the narrator’s voice recordings to achieve clarity and emotional depth. It also enhances foley and ambient effects, ensuring a rich, immersive soundscape that complements the visuals and narrative.
MY ROLE
In the creation of "Echoes of Us," my role transcended single departmental responsibilities. As Director, Visual Coordinator, and Video and Sound Editor, I was responsible for the holistic integration of hardware, software, and narrative structure, ensuring that the final interactive experience was emotionally cohesive and technically robust. My work focused on establishing the core artistic vision, defining the physical aesthetics, and sculpting the multimedia assets that would guide the participant through the father-daughter narrative.
Director: Shaping the Narrative Arc and Participant Journey
My primary responsibility as Director was to define the overall participant experience and narrative flow. The project is an exploration of memory, reflection, and potential reconciliation, and I ensured this emotional arc was mapped precisely to the physical objects. I guided the integration of the Arduino Nano and TouchDesigner to enforce the strict narrative sequence (baby shoes, volleyball, painting, chair). I designed the branching decision point that leads to one of three possible endings: indifference, regret, or reconciliation. This involved detailed work with the Flowchart logic, ensuring that the performer’s final actions correctly dictated the projected outcome. My directorial vision focused on maximizing the emotional weight of each interaction so that the performer felt fully immersed in the narrator's past.
Visual Coordinator: Blending Physicality and Projection
My duties as Visual Coordinator centered on marrying the physical installation with the projected digital content. This required careful planning of the environment and the selection of materials:
-
Object Selection and Rigging: I was responsible for selecting the five Story Objects (baby shoes, volleyball, painting, chair, and telephone), not just for their symbolic value, but for their ability to integrate seamlessly with the sensors (Tilt, Vibration, Distance, and Pressure). I ensured the objects retained their evocative, nostalgic feel despite the hidden technology.
-
Projection Mapping and Environment: I planned the use of the two 4k short-throw Projectors, ensuring the "memories" were vividly unfolded for the audience and that the different endings were displayed clearly. This involved coordinating the physical setup—lighting and background—to create the intimate, emotionally charged atmosphere visible in the installation images. * Sensory Feedback: I oversaw the use of WS2812B LED stripes to enhance the environment, ensuring the ambient lighting changed dynamically to complement the emotional tone of the narrator's current memory.
Video and Sound Editor: Sculpting the Memories
Perhaps the most time-intensive aspect of my role was the preparation and synchronization of the multimedia assets that formed the "memories" themselves. This required fluency in both video and audio post-production:
-
Video Editing and Syncing: I utilized DaVinci Resolve to edit the video clips of the memories (like the first steps or the volleyball game). Crucially, I synchronized these visuals precisely with the pre-recorded narration and Foley sound effects. The goal was to ensure the emotional delivery of the narrator’s voice, and the visual representation was perfectly timed, creating a seamless and polished experience for the audience.
-
Audio Enhancement and Soundscape Design: I employed Adobe Podcast to refine and enhance the audio elements. This included refining the narrator’s voice recordings to achieve clarity and emotional depth, and enhancing the Foley and ambient effects (such as the nostalgic soundtrack and the subtle sounds of scribbling on paper) to create a rich, immersive soundscape. The quality of the sound was vital, as it carried the bulk of the emotional narrative that the objects unlocked.
In conclusion, I ensured that "Echoes of Us" functioned as an evocative platform for shared memory by directing the overall experience, coordinating the visual interplay of light and object, and editing the precise timing of the video and sound.
Lessons Learned
Working on "Echoes of Us" provided profound insights into the challenges and triumphs of blending physical interactivity with rich, linear media. The lessons learned are rooted in the necessity of cross-disciplinary mastery and the primacy of the user's emotional journey over technical complexity:
1. The Hidden-Tech Principle: The success of the immersive narrative depended entirely on the invisibility of the technology. The Tilt, Vibration, Distance, and Pressure sensors needed to be perfectly hidden within the evocative Story Objects so that the magic of the memory unlocking remained intact. This confirmed that in narrative installations, the product designer’s first job is to conceal the mechanism, prioritizing the emotional connection over exposing the interface.
2. The Unbreakable Chain of Sync: The project's emotional impact hinged on the precise synchronization of digital assets. Any lag between the sensor trigger (the physical action) and the start of the edited audio/video (the emotional payoff) immediately broke the illusion. My intensive work in DaVinci Resolve and Adobe Podcast taught me that in experiential design, milliseconds matter, and the media timing must be as robust and responsive as the hardware triggers.
3. Ambiguity as an Asset: Designing the final branching narrative—the three possible endings—highlighted the value of providing meaningful, yet slightly ambiguous, choices to the performer. The lack of a clear "right" answer (indifference, regret, or reconciliation) forced genuine reflection, fulfilling the installation’s core purpose. This reinforced the directorial lesson that sometimes, the most powerful design choice is to offer complexity rather than simple satisfaction.