do

ATLAS researchers converge at TEI’26 to showcase their work on tangible, embedded, and embodied interaction

Michael Kwolek — Mon, 09 Mar 2026 15:06:26 +0000

ATLAS researchers converge at TEI’26 to showcase their work on tangible, embedded, and embodied interaction Michael Kwolek Mon, 03/09/2026 - 09:06

Michael Kwolek

Sound, vision, movement and touch—ATLAS researchers explore many different ways humans can interact with computers, collect and analyze data, and empower creative exploration.

Nearly a dozen current and former ATLAS lab members will participate in ACM TEI’26 in Chicago (March 8-11, 2026), the 20th annual conference presenting the latest results in tangible, embedded, and embodied interaction.

This year’s conference theme is “Tide + Tied”. Organizers note, “By becoming a venue to bring multi-folded 'Tides' across diverse, interdisciplinary fields, the conference aims to bring researchers, designers, and artists with different backgrounds and interests together to be 'Tied,' weaving the future of the TEI community together.”

ATLAS has been involved with the TEI conference since its early years, with Professor and ACME Lab director, Ellen Do, and ATLAS director Mark Gross both actively involved behind the scenes.

Do, who is a co-author on three papers and three works-in-progress accepted at TEI ‘26, explains, “Each one of the projects is a documentation of how researchers think about ideas and how to implement them and get them to fruition.”

She elaborates, “The conference is called Tangible, Embedded and Embodied Interaction, so a lot of work we're doing is beyond the screen. Things that we touch and put together.”

Papers

Sound of Kigumi: A Playful VR Joinery Adjustment with Hammering Sound Feedback

Kosei Ueda, Ellen Yi-Luen Do, Hironori Yoshida

Abstract: Traditional carpentry faces a critical shortage of skilled workers due to limited opportunities for potential apprentices to access onsite woodworking experience. Through expert interviews, we learned the importance of hammering sound to judge the precision in Kigumi assembly, as master carpenters rely on differences between “soft sound" and “sharp sound" without relying on visuals. This paper presents Sound of Kigumi (SoK), a playful VR system for inexperienced users to casually experience sound sensory skills through the loop of hammering and chiseling. In SoK, users listen to hammering sound in relation to tightness, assess the precision of their work, and return to chiseling for further adjustments. Furthermore, SoK implements pseudo-haptic feedback by visually modifying hammering resistance based on chiseling progress. Expert evaluation indicated SoK replicates the hammering process and serves as an effective introductory tool, and user feedback confirmed SoK provides an immersive woodworking experience and effective Kigumi learning.

The first prototype: The user observes and hammers two types of Kigumi - one correctly processed without visible gaps when hammered and one incorrectly processed that reveals gaps upon hammering.

Why (Not) ReacTIVision: Emerging Challenges and Opportunities for Building Tangible User Interfaces with Computer Vision Toolkits

Krithik Ranjan, S. Sandra Bae, Peter Gyory, Ellen Yi-Luen Do, Clement Zheng, Rong-Hao Liang

Abstract: Outdated Computer Vision (CV) toolkits for Tangible User Interfaces (TUI) have led to fragmented practices, diminished reproducibility, and reduced community support. This paper examines the past, present, and future trajectory of CV-TUI toolkits. First, our scoping review of ACM literature reveals a divergence between applications using the limited interactions of established toolkits like ReacTIVision and the fragmented, bespoke systems built for complex interactions, highlighting the need for advanced toolkits that enable accessible making. Second, we present proof-of-concept applications using the contemporary ArUco fiducial marker library. We demonstrate how accessible hardware, like a top-down camera and a flat-panel display, can support a comprehensive design space of tangible interactions beyond 2D manipulation, including 3D spatial interaction, multi-device interaction, and actuated tangibles within canonical applications. Finally, reflecting on our findings, we offer six suggestions for building next-generation CV-TUI toolkits. This study provides the TUI community with an updated perspective to inform future research.

Sensing touch input on tokens with a capacitive touchscreen: a) Each ArUco-marked knob is augmented with a vinyl-cut copper sheet pattern. b-c) The knob transfers finger-touch inputs to the touchscreen when users interact with the token.

HyperDance: Real-Time Vibrotactile Stimulation Feedback of Inter-Brain Connectivity in Partner Dance

Thiago Rossi Roque, Ruojia Sun, Ellen Yi-Luen Do, Grace Leslie

Abstract: Building on the growing interest in technology-supported dance practice, neural imaging offers novel opportunities to reveal dancers’ internal states and expand the possibilities for augmented, embodied interaction. Despite advances in social neuroscience, the exploration of dance through brain imaging remains limited by technical challenges. To overcome these barriers, we developed and validated a real-time vibrotactile biofeedback system based on inter-brain coupling (IBC) measures from tango dancers using a mobile, synchronous multi-brain EEG system. We first conducted an empirical study recording synchronized EEG and motion data to test whether behavioral synchronization enhances inter-brain coupling. Insights from this study informed the design of our tangible neurofeedback system, which experienced dancers evaluated. Our findings support the Synchronicity Hypothesis of Dance and demonstrate how embodied technologies can enhance collective dance practice. This work introduces a novel methodological and interaction paradigm, bridging neural measurement with wearable feedback for socially situated embodied experiences.

HyperDance enables real-time measurement and tactile feedback of inter-brain coupling during natural partner dance practice.

Art and Performance

Bioactuated Tapestry: Converging Textile Craft and Moisture-Responsive Biomaterials

Eldy S. Lazaro Vasquez; Viola Arduini; Etta W Sandry; Katerina Houser; Srujana Golla; Mirela Alistar

Abstract: Bioactuated Tapestry is an installation that explores how biomaterials and textile craft unfold multiple temporalities of interaction. Structured in three zones, the installation moves from milk-based bioplastic samples that change shape quickly when misted, to a Sample Book that documents iterations of bioplastic integration into weaving, to a woven tapestry that changes shape slowly in response to humidity in the surrounding space. Together, these zones demonstrate how interaction can emerge from material behavior shaped through biomaterial formulation and, when woven, through structure. The work foregrounds biomaterial agency, weaving, and situated sustainability grounded in sourcing, fabrication, and practices of care. Through this convergence of biodesign and textile craft, Bioactuated Tapestry aligns with the TEI theme of Resurgence and Convergence, highlighting how material-led practices reconnect material experimentation, environmental attunement, and embodied ways of knowing.

Learn More

Detail of Bioactuated Tapestry, showing colored casein-based bioplastic strips woven through black cotton yarns. Moisture causes the bioplastic to change shape, and the weave directs that change into curling.

Pictorials

Designing for the Leaky Body: Exploring Biomaterial Absorption as Body-Material Interaction

Viola Arduini; Eldy S. Lazaro Vasquez; Srujana Golla; Mirela Alistar

Abstract: Leaking bodies are often concealed or disregarded in both society and design. Likewise, bodily fluids are rarely leveraged as triggers for material interaction in HCI. In this pictorial, we investigate how fluid-responsive biomaterials can enable porous, expressive, and cyclical interactions co-shaped by the body. We focus on a milk-derived bioplastic with reversible shape-changing properties, examining fluid absorption as a meaningful design affordance. Our material-led approach contributes both formulation and fabrication methods of casein bioplastic; while autoethnographic inquiry with a lactating body informed the development of Leaky Body Maps and speculative garments that position leakage as a generative site of body-material interaction. This work contributes to the discourse of feminist and posthuman HCI by centering bodily permeability, material responsiveness, and the potential of designing with – rather than concealing – leaky bodies.

Garment prototype showing where casein-based bioplastic was placed based on a body leak map of possible milk leakage.

Works In Progress

ArUcoTUI: Software Toolkit for Prototyping Tangible Interactions on Portable Flat-Panel Displays with OpenCV

Rong-Hao Liang, Steven Houben, Krithik Ranjan, S. Sandra Bae, Peter Gyory, Ellen Yi-Luen Do, Clement Zheng

Abstract: Tangible User Interfaces (TUIs) that integrate digital information with physical interaction require specialized hardware and complex calibration, limiting their adoption in portable or mobile display systems. This paper introduces ArUcoTUI, a computer vision (CV) toolkit for prototyping tangible interactions on portable screens, leveraging standard cameras and the OpenCV library. ArUcoTUI uses ArUco fiducial markers to detect physical inputs. The software toolkit offers streamlined calibration, a signal processing pipeline, and a client application that translates tangible input into structured events for use in HCI applications. Using a conventional camera in a top-down setting with a flat-panel display, we demonstrate how this toolkit supports the development of interactive surface TUIs with advanced features, including 3D spatial interaction, multi-device interaction, and actuated tangibles within applications. We describe the software implementation, which utilizes accessible hardware to support the development of these tangible interactions. We provide the results of a preliminary evaluation with users, including design implications and suggestions for future research and development.

ArUcoTUI is a software toolkit for rapid TUI prototyping on portable screens. It uses standard cameras, OpenCV, and ArUco markers for real-time object tracking. We demonstrate the applicability using an overhead camera for a) multi-token music control, b) above-screen gesture detection, c) multi-display board games, and d) actuated data visualization using robots.

Rig-a-Doodle: Tangible Kit for Dynamic Hand-drawn Character Animation

Krithik Ranjan, Khushbu Kshirsagar, Harrison Jesse Smith, Ellen Yi-Luen Do

Abstract: Character animation remains challenging for novices and children despite advances in digital tools. While recent tangible interfaces have lowered barriers by enabling creators to animate their drawings on paper, they are limited to preset animation sequences and support for only human-like characters. We present Rig-a-Doodle, a tangible kit and web application for fully open-ended character rigging animation, where creators can draw any character and construct a custom physical rig using everyday materials to animate it. This work-in-progress contributes a system of tangible interaction to animate hand-drawn characters by direct physical manipulation of custom rigs in real-time. We share findings from a preliminary workshop with adults to explore the kinds of expressive animation the kit enables, discover issues with interaction, and source ideas for future directions.

(Top-left) Rig-a-Doodle character template to draw the character and cut out CV markers for the rig. (Top-right, bottom-left, bottom-right) The three steps of Capture, Assign, and Play illustrated with screenshots from the Rig-a-Doodle application.

Lighting the Reef: Modular Paper Circuits as Ecological Metaphor

Ruhan Yang, Yuchen Zhang, Ellen Yi-Luen Do

Abstract: We present Lighting the Reef, an interactive installation that uses modular 3D paper circuits to explore ecological fragility. Participants build coral structures from foldable paper blocks with copper tape and low-voltage components. When connections align, coral modules glow, metaphorically expressing the energy exchange between coral and zooxanthellae, the symbiotic algae crucial to coral metabolism. Pollution modules add resistance that dims light or interrupts the current entirely, mirroring environmental disruption. We position Lighting the Reef as a Research through Design case that articulates fragility as an interaction aesthetic and ecological metaphor. We reflect on how modular circuitry, material constraints, and embodied play make precarity tangible. We also report workshops with 15 participants that discussed themes of care, collapse, and interdependence. We contribute insights into designing for fragility with modular circuits, ecological storytelling through tangible interaction, and accessible and reproducible designs for participatory sustainability education.

Lighting the Reef is a tangible installation built from 3D paper circuit modules, whose illumination depends on alignment and balance. As participants assemble and adjust the blocks, the lights brighten, dim, or turn off, reflecting the changing conditions of the coral system.

Members of several ATLAS labs show off the latest research on human-computer interactivity.

Off

Traditional

White

ATLAS community presents new research on interactive systems at DIS 2025

Michael Kwolek — Thu, 26 Jun 2025 17:14:27 +0000

ATLAS community presents new research on interactive systems at DIS 2025 Michael Kwolek Thu, 06/26/2025 - 11:14

The 2025 ACM Designing Interactive Systems Conference (DIS) in Madeira, Portugal, features work from ten ATLAS community members representing three labs. This year’s event has five focus areas: Critical Computing and Design Theory, Design Methods and Processes, Artifacts and Systems, Research Through Design, and AI and Design with an overall theme around “design that transcends human-centered perspectives.”

ATLAS researchers study a broad range of topics, from human-computer interaction to biomaterials to woven forms.

Ellen Do, professor and ACME director, explains what connects the work our community is presenting at the conference: “I think all of the papers and presentations we have are on designing interactive systems. Some of the systems could be physical, some could be digital, some could be human-and-people, human-and-physical objects. So I think the theme about interactive systems and how you make systems interactive, what kind of user experience or human experience or immersive experience with the object or system or even the ecosystem, or the human communication system—I think that's all there.”

ATLAS research at DIS 2025

"Chaotic, Exciting, Impactful": Stories of Material-led Designers in Interdisciplinary Collaboration
Gabrielle Benabdallah, Eldy S. Lazaro Vasquez (ATLAS PhD student), Laura Devendorf (ATLAS Unstable Design Lab director, associate professor), Mirela Alistar (ATLAS Living Matter Lab director, assistant professor)

This paper explores the dynamics of interdisciplinary collaboration between designers, scientists, and engineers through ten stories as told from the perspective of material-led designers. These stories focus on material-led designers working in contexts like biodesign and smart textiles, where novel materials, fabrication methods, and technology often intersect, requiring cross-disciplinary collaboration. By including perspectives from designers within and adjacent to HCI, the study broadens the understanding of interdisciplinary teamwork that combines scientific, technical, and craft-based expertise. Our analysis highlights how designers navigate challenges like differing terminologies, epistemic hierarchies, and conflicting priorities. We discuss strategies such as material prototypes, attitudes of inquiry and openness, switching lexicons, and the value of interdisciplinary contexts. This research underscores designers as “translators” who mediate epistemological tensions, use tangible artifacts to communicate, and articulate possible applications. This research contributes ten stories as narrative resources for understanding strategies and fostering interdisciplinary spaces within HCI.

Towards Yarnier Interactive Textiles: Mapping a Design Journey through Hand Spun Conductive Yarns
Etta W. Sandry (ATLAS PhD student), Lily M. Gabriel (ATLAS undergraduate student), Eldy S. Lazaro Vasquez (ATLAS PhD student), Laura Devendorf (ATLAS Unstable Design Lab Director, associate professor)

The ability to create a wide and varied set of interactive textiles depends on the materials that one has available. Currently, the range of yarns that can be used to bring interactivity to textiles is greatly limited, especially considering the diversity available in non-conductive yarns. This pictorial traces a design journey into hand spinning that seeks to address this limitation and contributes samples of techniques and materials that could be used to create conductive yarns along with reflection on design methods that enabled us to explore a wider range of aesthetic expressions. We advocate for an approach that reconnects with the textiles in e-textiles, embraces divergence, and prioritizes the material as the driver of a design concept. We offer pathways for readers and researchers to continue this exploration within varied domains and practices.

Connect! A Circuit-Driven Card Game
Ruhan Yang (ATLAS PhD alum), Ellen Yi-Luen Do (ATLAS ACME Lab director, professor)

Hybrid physical-digital games often rely on screen-based interactions, which can detract from their tactile nature. We introduce Connect!, a card game that integrates paper circuits and real-time LED feedback, enabling players to construct functional circuits as part of gameplay. Unlike traditional hybrid games, Connect! embeds feedback directly into physical components while preserving material interaction. We conducted a user study comparing gameplay with and without electronic feedback. Our findings suggest that real-time feedback not only increased engagement but also altered players' behavior, encouraging rule exploration and emergent play. Our work contributes to tangible interaction and game-based learning, demonstrating the potential of low-cost electronics in enhancing interactive experiences.

Connect! game cards

From Data to Discussion: Interfaces for Collective Inquiry and Open-Ended Data Creation
David Hunter (ATLAS PhD student)

Data can enrich our understanding of the world and improve our society. However the datafication of our society comes with challenges for empowering communities. In designing systems for recording and representing data, a theme has emerged of these interfaces as the site of conversations and sense-making, and the participatory nature is valuable beyond the data itself. This insight has led me to investigate tools and experiences that enable open-ended data creation and exploration as a grounding for discussion and prompting action. The goal is to design interfaces and systems for exploring places and futures through data, to empower communities and supporting civic participation, learning and making, situational awareness, and scenario planning. In this pictorial I present five ongoing research projects investigating these ideas.

How to Data Walk

Knitting with unknown trees: assembling a more-than-human practice
Doenja Oogjes, Ege Kökel, Netta Ofer (ATLAS PhD alum), Hsiang-Lin Kuo, Jasmijn Vugts, Troy Nachtigall, Torin Hopkins (ATLAS PhD alum)

In this pictorial, we explore alternative ways of knowing urban trees through a more-than-human lens. Using a municipal tree dataset, we focus on “unknown” trees—entries unclassified due to error, decay, or absence—highlighting the limits of quantification and fixed knowledge systems. Urban trees, while critical for ecosystems, are often shaped by technological interventions (e.g., GIS, IoT sensors, AI diagnostics) that prioritize their utility over other expressions. We engage in knitting as a material inquiry to foreground nonhuman agencies and relational entanglements. Through reflective shifts and compromises, this project questions normative design practices, seeking to amplify nonhuman participation. We make two contributions. Firstly, we offer insights into fostering alternative, relational engagements with urban ecologies. Secondly, we reflect on our process of surfacing and working with agentic capacities, articulating guidance for other design researchers. Through this, we advocate for fragmented approaches that embrace complicity and complexity in more-than-human design.

Designing Interfaces that Support Temporal Work Across Meetings with Generative AI
Rishi Vanukuru (ATLAS PhD student), Payod Panda, Xinyue Chen, Ava Elizabeth Scott, Lev Tankelevitch, Sean Rintel

Temporal work is an essential part of the modern knowledge workplace, where multiple threads of meetings and projects are connected across time by the acts of looking back (retrospection) and ahead (prospection). As we develop Generative AI interfaces to support knowledge work, this lens of temporality can help ground design in real workplace needs. Building upon research in routine dynamics and cognitive science, and an exploratory analysis of real recurring meetings, we develop a framework and a tool for the synergistic exploration of temporal work and the capabilities of Generative AI. We then use these to design a series of interface concepts and prototypes to better support work that spans multiple scales of time. Through this approach, we demonstrate how the design of new Generative AI tools can be guided by our understanding of how work really happens across meetings and projects.

Members of three ATLAS labs show how interactive technology can create possibilities for new means of productivity, data analysis, creativity and play.

Off

Traditional

White

Colorado-based Computer Graphics Professionals Make Their Mark at SIGGRAPH 2024

Anonymous — Fri, 02 Aug 2024 16:30:29 +0000

Colorado-based Computer Graphics Professionals Make Their Mark at SIGGRAPH 2024 Anonymous (not verified) Fri, 08/02/2024 - 10:30

ATLAS community members, including professor Ellen Do and PhD student Ruhan Yang, presented at this year's conference in Denver.

Off

Traditional

White

Public-private partnership drives attention for ATLAS research in augmented and mixed reality

Anonymous — Thu, 18 Jul 2024 16:41:59 +0000

Public-private partnership drives attention for ATLAS research in augmented and mixed reality Anonymous (not verified) Thu, 07/18/2024 - 10:41

Michael Kwolek

Partnerships between universities and industry can yield important research and commercial breakthroughs. ATLAS professor Ellen Do has worked to cultivate relationships between CU ��ý�� and industry players, including as a member of the Pervasive Personalized Intelligence (PPI) Center, to support graduate students and enhance opportunities for commercialization of ATLAS research.

The PPI Center, which recently concluded its tenure, was founded “with a mission of bringing industry and university talent together to solve the intelligence challenges faced by software and computer engineers in Internet of Things systems." It operated under the supervision of the National Science Foundation and included members from NEC, Intel and Trimble.

“It’s been such a good experience. We’ve learned a lot. Ellen Do and her team have helped to expand our thinking and encouraged us to explore new areas.” - Dr. Haifeng Chen, Head of Data Science Department at NEC Laboratories, and his colleague Kai Ishikawa, Principal Researcher (PPI Center event recap)

The PPI Center’s Spring 2024 Industry Advisory Board Meeting in Portland, OR, included a research poster session, and ATLAS students were honored with three of the four awards industry attendees voted on at the event.

Suibi Che-Chuan Weng, PhD student, won "Most Industry Ready" for Editing Reality: Empowering Users to Manipulate Reality through Addition, Erasing, and Modification with Speech to Prompt in Mixed Reality.
Rishi Vanukuru, PhD student, won "Most Impactful" for Asynchronous spatial guidance using mobile devices and Augmented Reality.
Ada Zhao, MS student, won "Most Impactful" for The WizARd and Apprentice: Augmented Reality Expert Capture for Training Novices.

2 more ATLAS PhD students participated: Krithik Ranjan presented PuppetGuide: Tangible Personalized Museum Tour Guides using LLMs and David Hunter presented Tangible Interaction with Object Detection and Large Language Models.

As for the experience participating in the PPI Center, Do says, “it is good to know that the industry is interested in supporting research and considers our research relevant.” She sees ways ATLAS could form partnerships within several industry sectors on a range of themes due to the multidisciplinary nature of the research conducted here.

Since their involvement in PPI started, Do and her team have had a series of meetings with mentors from global technology firms, discussing collaborative research opportunities.

Vanukuru is currently doing an internship at Microsoft Research Cambridge focused on spatial computing in its VR/AR group. Weng and Zhao are working on research in the ACME Lab this summer, extending the Editing Reality (and PuppetGuide), and WizARd and Apprentice projects with interns from the CU SPUR program. Zhao is also conducting a pilot study, interviewing laser cutter operating experts about how they would demonstrate operations and how they can annotate their demonstration using the WizARd prototype for novice learners. Hunter has embarked on an internship with Trimble this summer, while he and Ranjan are also working in the ACME Lab.

ACME Lab members built relationships with industry players through the Pervasive Personalized Intelligence (PPI) Center by collaborating on solutions to challenges in building Internet of Things systems. Three ATLAS PhD students took home awards from the PPI Center's Spring 2024 Advisory Board Meeting.

Off

Traditional

White

ATLAS in Ireland: 12 community members present at TEI’24

Anonymous — Fri, 09 Feb 2024 19:05:23 +0000

ATLAS in Ireland: 12 community members present at TEI’24 Anonymous (not verified) Fri, 02/09/2024 - 12:05

Michael Kwolek

ATLAS is well-represented at #TEI2024 - the 18th ACM International Conference on Tangible, Embedded and Embodied Interaction. This year’s conference, in Cork, Ireland, celebrates “cutting-edge scientific research and art that is on the edge of disciplines and on the edge of new unique developments and possibilities.”

Research from 12 members of the ATLAS community including faculty, alumni and students is featured at the conference. The work spans a range of disciplines, including weaving, biomaterials, mixed reality and robotics. In addition, ACME Lab director, Ellen Do, acted as Co-Chair of Graduate Student Consortium; PhD student, Sandra Bae, was an Associate Chair for Pictorials; and ATLAS PhD alum, Fiona Bell, was an Associate Chair for Papers.

Research ATLAS PhD students presented at TEI’24

Loom Pedals: Retooling Jacquard Weaving for Improvisational Design Workflows
Shanel Wu, Xavier A Corr, Xi Gao, Sasha De Koninck, Robin Bowers, and Laura Devendorf

Abstract: We present the Loom Pedals, an open-source hardware/software interface for enhancing a weaver’s ability to create on-the-fly, improvised designs in Jacquard weaving. Learning from traditional handweaving and our own weaving experiences, we describe our process of designing, implementing, and using the prototype Loom Pedals system with a TC2 Digital Jacquard loom. The Loom Pedals include a set of modular, reconfigurable foot pedals which can be mapped to parametric Operations that generate and transform digital woven designs. Our novel interface integrates design and loom control, providing a customizable workflow for playful, improvisational Jacquard weaving. We conducted a formative evaluation of the prototype through autobiographical methods and collaboratively developed future Loom Pedals features. We contribute our prototype, design process, and conceptual reflections on weaving as a human-machine dialog between a weaver, the loom, and many other agents.

Bio-Digital Calendar: Attuning to Nonhuman Temporalities for Multispecies Understanding
Fiona Bell, Joshua Coffie, and Mirela Alistar

Abstract: We explore how actively engaging with the temporalities of a nonhuman organism can lead to multispecies understanding. To do so, we design a bio-digital calendar that brings attention to the growth and health of kombucha SCOBY, a symbiotic culture of bacteria and yeast that lives in a tea medium. The non-invasive bio-digital calendar surrounds the kombucha SCOBY to track (via sensors) and enhance (via sound) its growth. As we looked at and listened to our kombucha SCOBY calendar on a daily basis, we became attuned to the slowness of kombucha SCOBY. This multisensory noticing practice with the calendar, in turn, destabilized our preconceived human-centered positionality, leading to a more humble, decentered relationship between us and the organism. Through our experiences with the bio-digital calendar, we gained a better relational multispecies understanding of temporalities based on care, which, in the long term, might be a solution to a more sustainable future.

Wizard of Props: Mixed Reality Prototyping with Physical Props to Design Responsive Environments
Yuzhen Zhang, Ruixiang Han, Ran Zhou, Peter Gyory, Clement Zheng, Patrick C. Shih, Ellen Yi-Luen Do, Malte F Jung, Wendy Ju, and Daniel Leithinger

Abstract: Driven by the vision of future responsive environments, where everyday surroundings can perceive human behaviors and respond through intelligent robotic actuation, we propose Wizard of Props (WoP): a human-centered design workflow for creating expressive, implicit, and meaningful interactions. This collaborative experience prototyping approach integrates full-scale physical props with Mixed Reality (MR) to support ideation, prototyping, and rapid testing of responsive environments. We present two design explorations that showcase our investigations of diverse design solutions based on varying technology resources, contextual considerations, and target audiences. Design Exploration One focuses on mixed environment building, where we observe fluid prototyping methods. In Design Exploration Two, we explore how novice designers approach WoP, and illustrate their design ideas and behaviors. Our findings reveal that WoP complements conventional design methods, enabling intuitive body-storming, supporting flexible prototyping fidelity, and fostering expressive environment-human interactions through in-situ improvisational performance.

Making Biomaterials for Sustainable Tangible Interfaces
Fiona Bell, Shanel Wu, Nadia Campo Woytuk, Eldy S. Lazaro Vasquez, Mirela Alistar, and Leah Buechley

Abstract: In this studio, we will explore sustainable tangible interfaces by making a range of biomaterials that are bio-based and readily biodegradable. Building off of previous TEI studios that were centered around one specific biomaterial (i.e., bioplastics at TEI’22 and microbial cellulose at TEI’23), this studio will provide participants the ability to experience a wide variety of biomaterials from algae-based bioplastics, to food-waste-based bioclays, to gelatin-based biofoams. We will teach participants how to identify types of biomaterials that are applicable to their own research and how to make them. Through hands-on activities, we will demonstrate how to implement biomaterials in the design of sustainable tangible interfaces and discuss topics sensitized by biological media such as more-than-human temporalities, bioethics, care, and unmaking. Ultimately, our goal is to facilitate a space in which HCI researchers and designers can collaborate, create, and discuss the opportunities and challenges of working with sustainable biomaterials.

Paper Modular Robot: Circuit, Sensation Feedback, and 3D Geometry
Ruhan Yang

Abstract: Modular robots have proven valuable for STEM education. However, modular robot kits are often expensive, which makes them limited in accessibility. My research focuses on using paper and approachable techniques to create modular robots. The kit’s design encompasses three core technologies: paper circuits, sensation feedback mechanisms, and 3D geometry. I have developed proof-of-concept demonstrations of technologies for each aspect. I will integrate these technologies to design and build a paper modular robot kit. This kit includes various types of modules for input, output, and other functions. My dissertation will discuss the development of these technologies and how they are integrated. This research will address the considerations and techniques for paper as an interactive material, providing a guideline for future research and development of paper-based interaction.

Research from 12 members of the ATLAS community including faculty, alumni and students is featured at the 18th ACM International Conference on Tangible, Embedded and Embodied Interaction.

Off

Traditional

White

Ellen Yi-Luen Do Presents Keynote on Fun with Creative Technology & Design at TaiCHI 2023

Anonymous — Wed, 13 Sep 2023 18:51:39 +0000

Ellen Yi-Luen Do Presents Keynote on Fun with Creative Technology & Design at TaiCHI 2023 Anonymous (not verified) Wed, 09/13/2023 - 12:51

Michael Kwolek

ATLAS Professor Ellen Yi-Luen Do had the opportunity to be a keynote speaker at TaiCHI 2023, a symposium hosted by the Taiwan Human-Computer Interaction Society at Taiwan University in Taipei. The event gathered researchers and practitioners across a range of backgrounds in technology, design and human factors to deepen community connections and explore new ideas.

Sessions included presentations on fabrication, perception, interactions and other timely topics, with a surprising range in mediums from humble materials like felt and puppets to advanced VR technologies and metaverse interactivity.

As director of the ACME Lab at ATLAS, Do and her team conduct research on using everyday items as interfaces, creating objects to think with, new ways of working, and methods and tools to help others make things. Do delivered her presentation, entitled “Fun with Creative Technology & Design”, advocating for playful computing with easily accessible materials like paper and cardboard, while highlighting ways to make toolkits for others to create for themselves.

The audience, which included experts in computer science, psychology, media, art, design and business responded enthusiastically, finding common ground in this relatable, inclusive approach to otherwise complex technologies. Do received a particularly warm reception from students in the field. She noted, “Several students came to thank me for my talk, stating that they learned so much from me, and that they never thought research could be this fun and interesting.”

Do expressed excitement for a few standout presentations from the conference including Ed Chi, Distinguished Scientist at Google DeepMind, who delivered a keynote on the large language model revolution. She said, “I was happy to learn that Bard will be a tool-use application applying to many of the Google apps and services people already use, including Maps, Sheets, Gmail, Docs, and more.”

She also called out FeltingReel: Density Varying Soft Fabrication with Reeling and Felting by Ping-Yi Wang and Lung-Pan Cheng as particularly intriguing.

Back in 2015, Do wrote the article “A flourishing field: a guide to HCI in China, Taiwan, and Singapore”, and saw the founding of Taiwan HCI. Looking back, she reflects, “I’m happy to see TaiCHI 2023 have 300 people registered with vibrant discussions, demos and posters. It's definitely growing!”

ATLAS Professor Ellen Yi-Luen Do presented on Fun with Creative Technology & Design as keynote speaker at TaiCHI 2023.

Off

Traditional

White

16 Members of the ATLAS Community Present Groundbreaking Research on Human-Computer Interaction at ACM DIS 2023

Anonymous — Wed, 05 Jul 2023 19:43:45 +0000

16 Members of the ATLAS Community Present Groundbreaking Research on Human-Computer Interaction at ACM DIS 2023 Anonymous (not verified) Wed, 07/05/2023 - 13:43

Michael Kwolek

ATLAS is well-represented at this year’s ACM Designing Interactive Systems (DIS) 2023 conference convening at Carnegie Mellon University in Pittsburgh from July 10-14, 2023. This year’s theme is resilience.

"Resilience is at once about flexibility, durability, and strength as well as a sense of mutuality and hope where solidaristic modes of engagement make new kinds of worlds possible.

This also recognizes that resilience takes many forms in design discourse, ranging across: indigenous knowledge, more-than-human perspectives, and the relationship between human, material and artificial intelligences."

It is exciting to see members across more than half of ATLAS labs represented in this year’s proceedings, with broad-reaching research covering microbiomes as materials for interactive design; 3D printing with spent coffee grounds; personal informatics systems; improving cross-disciplinary collaboration among artists and researchers; expressive movement for altering emotions and awareness; and the intersection of crocheting and data. Take a look:

µMe: Exploring the Human Microbiome as an Intimate Material for Living Interfaces
Fiona Bell (ATLAS PhD alum), Michelle Ramsahoye (ATLAS affiliate PhD student), Joshua Coffie (ATLAS MS alum), Julia Tung (ATLAS BS student), and Mirela Alistar (ATLAS Living Matter Lab director, assistant professor)

Our bodies are home to an unseen ecosystem of microbes that live in symbiosis with us. In this work, we extend the “human” in Human-Computer Interaction (HCI) to include these microbes. Specifically, we explore the skin microbiome as an intimate material for interaction design. Viewing the body as a microbial interface, we start by presenting a method to grow our microbiome such that it becomes visible to the human eye. We then present a design space that explores how different environmental parameters, such as temperature and growth media, can be controlled to influence the color of the microbiome. We further investigate how our interactions in a daily uncontrolled environment (e.g., exercising, hugging, typing) can impact the microbiome. We demonstrate several wearable applications that reveal and control the microbiome. Lastly, we address the challenges and opportunities of working with the microbiome as an intimate, living material for interaction design.

Designing a Sustainable Material for 3D Printing with Spent Coffee Grounds
Michael L. Rivera (ATLAS Utility Research Lab Director, assistant professor), S. Sandra Bae (ATLAS PhD student)

The widespread adoption of 3D printers exacerbates existing environmental challenges as these machines increase energy consumption, waste output, and the use of plastics. Material choice for 3D printing is tightly connected to these challenges, and as such researchers and designers are exploring sustainable alternatives. Building on these efforts, this work explores using spent coffee grounds as a sustainable material for prototyping with 3D printing. This material, in addition to being compostable and recyclable, can be easily made and printed at home. We describe the material in detail, including the process of making it from readily available ingredients, its material characteristics and its printing parameters. We then explore how it can support sustainable prototyping practices as well as HCI applications. In reflecting on our design process, we discuss challenges and opportunities for the HCI community to support sustainable prototyping and personal fabrication. We conclude with a set of design considerations for others to weigh when exploring sustainable materials for 3D printing and prototyping.

For additional details, see our article on how this and other Utility Research Lab projects won awards at the Rocky Mountain RepRap Festival.

Being, Having, Doing, and Interacting: A Personal Informatics Approach to Understanding Human Need Satisfaction in Everyday Life
Michael Jeffrey Daniel Hoefer, Stephen Voida, (ATLAS affiliate assistant professor, founding faculty, information science)

A grand challenge for computing is to better understand fundamental human needs and their satisfaction. In this work, we design a personal informatics technology probe that scaffolds reflection on how time-use satisfies Max-Neef's fundamental needs of being, having, doing, and interacting via self-aspects, relationships and organizations, activities, and environments. Through a combination of a think-aloud study (N=10) and a week-long in situ deployment (N=7), participants used the probe to complete self- aspect elicitation and Day Reconstruction Method tasks. Participants then interacted with network visualizations of their daily lives, and discovered insights about their lives. During the study, we collected a dataset of 662 activities annotated with need satisfaction ratings. Despite challenges in operationalizing a theory of need through direct elicitation from individuals, personal informatics systems show potential as a participatory and individually meaningful approach for understanding need satisfaction in everyday life.

Enhancing Accessibility in Soft Robotics: Exploring Magnet-Embedded Paper-Based Interactions
Ruhan Yang (ATLAS PhD student), Ellen Yi-Luen Do (ATLAS ACME Lab director, faculty member)

This paper explores the implementation of embedded magnets to enhance paper-based interactions. The integration of magnets in paper-based interactions simplifies the fabrication process, making it more accessible for building soft robotics systems. We discuss various interaction patterns achievable through this approach and highlight their potential applications.

[Workshop] Towards Mutual Benefit: Reflecting on Artist Residencies as a Method for Collaboration in DIS
Laura Devendorf (ATLAS Unstable Design Lab director, assistant professor), Leah Buechley, Noura Howell, Jennifer Jacobs, Hsin-Liu (Cindy) Kao, Martin Murer, Daniela Rosner, Nica Ross, Robert Soden, Jared Tso, Clement Zheng (ATLAS PhD alum)

While cross-disciplinary collaboration has long been, and continues to be a cornerstone of inventive work in interactive design, the infrastructures of academia, as well as barriers to participation imposed by our professional organizations, make collaboration for some groups harder than others. In this workshop, we’ll focus specifically on how artists residencies are addressing (or not) the challenges that artists, craftspeople, and/or independent designers face when collaborating with researchers affiliated with DIS. While focusing on the question “what is mutual benefit”, this workshop seeks to combine the perspectives of artists as well as researchers collaborating with artists (through residencies or otherwise) to (1) reflect on benefits or deficiencies in what we are currently doing and (2) generate resources for our community to effectively structure and evaluate our methods of collaboration with artists. Our hope is to provide recognition of and pathways for equitable inclusion of artists as a first step towards broader infrastructural change.

Refer to the Unstable Design Lab website for more details on this research.

[Demo] SoniSpace: Expressive Movement Interaction to Encourage Taking Up Space with the Body
Ruojia Sun (ATLAS PhD student), Althea Vail Wallop (ATLAS MS student), Grace Leslie (ATLAS Brain Music Lab director, assistant professor), Ellen Yi-Luen Do (ATLAS ACME Lab director, faculty member)

Movement forms the basis of our thoughts, emotions, and ways of being in the world. Informed by somaesthetics, we design for "taking up space" (e.g. encouraging expansive body movements), which may in turn alter our emotional experience. We demonstrate SoniSpace, an expressive movement interaction experience that uses movement sonification and visualization to encourage users to take up space with their body. We use a first-person design approach to embed qualities of awareness, exploration, and comfort into the sound and visual design to promote authentic and enjoyable movement expression regardless of prior movement experience. Preliminary results from 20 user experiences with the system show that users felt more comfortable with taking up space and with movement in general following the interaction. We discuss our findings about designing for somatically-focused movement interactions and directions for future work.

[Demo] Crochet and Data Activity Book
Mikhaila Friske (ATLAS affiliate PhD student)

This demo focuses around crocheting and data. In addition to a physical workbook for conference goers to peruse and try, there will be a few small set-ups for specific activities and a small craft circle for people to craft within if they so choose.

Off

Traditional

White

ATLAS Members Explore Childhood Play and Learning Through Interactive Design at IDC 2023

Anonymous — Mon, 19 Jun 2023 18:29:52 +0000

ATLAS Members Explore Childhood Play and Learning Through Interactive Design at IDC 2023 Anonymous (not verified) Mon, 06/19/2023 - 12:29

11 ATLAS community members have contributed to work featured at the 22nd annual ACM Interaction Design and Children (IDC) Conference to be held on June 19-23, 2023 at Northwestern University in Chicago, Illinois. IDC is the premier international conference for researchers, educators and practitioners to share the latest research findings, innovative methodologies and new technologies in the areas of inclusive child-centered design, learning and interaction. IDC’23 is hosted by the Center for Computer Science and Learning Sciences at Northwestern University.

Coming out of the pandemic, this year’s theme asks participants to “rediscover childhood” to understand what it means to be a child in this and coming decades and what adults can do to provide a sustainable and equitable future for the next generation. Key topics include privacy, ethics, equity, social and emotional wellbeing, sustainability, and healthy human development.

Research presented by ATLAS faculty, students and affiliates

Designing Together, Miles Apart: A Longitudinal Tabletop Telepresence Adventure in Online Co-Design with Children
Casey Lee Hunt (ATLAS THING Lab member, PhD student), Kaiwen Sun, Zahra Dhuliawala, Fumi Tsukiyama, Iva Matkovic, Zachary Schwemler (ATLAS MS alumnus), Anastasia Wolf, Zihao Zhang, Allison Druin, Amanda Huynh, Daniel Leithinger (ATLAS THING Lab Director, Computer Science faculty member), Jason Yip

Children’s online co-design has become prevalent since COVID-19. However, related research focuses on insights gained across several shorter-term projects, rather than longitudinal investigations. To explore longitudinal co-design online, we engaged in participatory design with children (ages 8 - 12) for 20 sessions in two years on a single project: an online collaboration platform with tabletop telepresence robots. We found that (1) the online technology space required children to play a role as technology managers and troubleshooters, (2) the home setting shaped online social dynamics, and (3) providing children the ability to choose their design techniques prevented gridlock from situational uncertainties. We discuss how each finding resulted from interplay between our long-term technology design and online co-design processes. We then present insights about the future of online co-design, a conceptual model for longitudinal co-design online, and describe opportunities for further longitudinal online co-design research to generate new methods, techniques, and theories.

Exploring Computational Thinking with Physical Play through Design
Junnan Yu, Ronni Hayden (PhD student), Ricarose Roque (Assistant Professor, Information Science)

Physical play has often been leveraged to provide children with active and engaging learning experiences. However, coding activities are predominantly sedentary in front of the screen, and the application of physical play in Computer Science education is less explored, e.g., how can we engage in computational thinking (CT) through physical play? In this design-based exploration, we conducted three design activities where young children, college students, and researchers were invited to create physical play projects using the BBC micro:bit and reflect on their experiences. By examining participants’ projects and creating processes, we provide empirical evidence that remixing physical play activities with coding can engage learners in various CT concepts and practices, reveal how CT concepts and practices can be represented in physical play, and highlight implications for designing physical play-mediated computational learning experiences. Ultimately, we encourage more learning experiences to incorporate physical play into computing education for children.

Ricarose Roque chairs the session “Computational and Data Literacy” in which this paper is included.

[Pictorial] Imagining Alternative Visions of Computing: Photo-Visuals of Material, Social, and Emotional Contexts from Family Creative Learning
Ricarose Roque (Assistant Professor, Information Science)

This pictorial presents visuals of families engaging with creative technologies as “knowledge-building artifacts” to provoke reflection on the social, material, and emotional context of designed interactions (“things that make you think”) as well as provocations to re-value these contexts and promote alternative visions in what and how engagement with computing can look like (“things that matter”). The selected images are from a large and ongoing collection of documentation from a family technology program. The images were captured using the Reggio Emilia documentation approach to documentation, which aims to “make learning visible.”

Ricarose Roque is one of three Pictorial Chairs in the conference Organizing Committee.

[Work-in-progress] Cartoonimator: A Low-cost, Paper-based Animation Kit for Computational Thinking
Krithik Ranjan (ATLAS ACME Lab member, PhD student), Peter Gyory (ATLAS ACME Lab member, PhD Candidate), Michael L. Rivera (Utility Research Lab Director, Assistant Professor, Human-Computer Interaction and Digital Fabrication), and Ellen Yi-Luen Do (ATLAS ACME Lab Director, Computer Science faculty member)

Computational thinking has been identified as an important skill for children to learn in the 21st century, and many innovative kits and tools have been developed to integrate it into children’s learning. Yet, most solutions require the use of devices like computers or other expensive hardware, thus being inaccessible to low-income schools and communities. We present Cartoonimator, a low-cost, paper-based computational kit for children to create animations and engage with computational thinking. Cartoonimator requires only paper and a smartphone to use, offering an affordable learning experience. Children can draw the scenes and characters for their animation on the paper, which is printed with computer vision markers. We developed the mobile web app to provide an interface to capture keyframes and compile them into animations. In this paper, we describe the implementation and workflow of Cartoonimator, its deployment with children at a local STEAM event, and a planned evaluation for the kit.

[Work-in-progress] Empower Children in Nigeria to Design the Future of Artificial Intelligence (AI) through Writing
Cornelius Onimisi Adejoro, Luise Arn, Larissa Schwartz (Master's student), Tom Yeh (Associate Professor, Computer Science)

This paper presents a new approach to engaging children in Nigeria to share their views of AI. This approach is centered on an inclusive writing contest for children in a secondary school in Abuja to write about AI to compete for prizes and share their writings with others. A preliminary analysis of the first 11 articles we received exhibits diverse gender and ethnic representation that conveys cultural values and perspectives distinct from those of the children in Western countries. This finding suggests future work to conduct an in-depth cross-cultural analysis of the articles and to replicate similar writing contests to engage children in other underrepresented countries

Off

Traditional

White

ATLAS affiliates receive seed grants to study AI-augmented learning

Anonymous — Wed, 24 May 2023 17:12:39 +0000

ATLAS affiliates receive seed grants to study AI-augmented learning Anonymous (not verified) Wed, 05/24/2023 - 11:12

The Engineering Education and AI-Augmented Learning Interdisciplinary Research Theme awarded multiple seed grants this spring to help spur research teaming in the college and boost early projects with the high potential for societal impact, including to several ATLAS Institute affiliates.

Off

Traditional

White

ACME Lab Champions Humble Materials for Innovative Human-Computer Interactions at CHI 2023

Anonymous — Mon, 22 May 2023 16:39:13 +0000

ACME Lab Champions Humble Materials for Innovative Human-Computer Interactions at CHI 2023 Anonymous (not verified) Mon, 05/22/2023 - 10:39

Over the years, the computer-human interaction field has seen many trends. For a time, gesture and pen-based interactions were key, then with the rising ubiquity of smartphones came a focus on haptic technologies. Now according to Ellen Do, ATLAS ACME Lab director, “material exploration” was the theme of the day at CHI 2023, the premiere conference on computer-human interaction.

Where researchers tend to focus on advanced electronics and innovative fibers, Do champions the use of everyday components as the basis for radical creativity and invention.

At the ACME Lab, Do notes that her students consistently look to humble, easy-to-source materials like paper, cardboard and even couch cushion foam for inspiration. These ubiquitous materials are often overlooked, but they offer many possibilities for developing ingenious tools for human-computer interaction that can be printed, cut and assembled with ease.

Access to off-the-shelf tools allows ACME Lab researchers to speed the prototyping process, giving them freedom to quickly iterate and push their thinking. When paired with the advanced camera and sensor technology found in everyday smartphones and off-the-shelf electronics like microprocessors, the team has the power to create limitless forms of human-computer interaction.

The team discovered that paper markers make a viable alternative to physical circuits, requiring fewer tools and expertise to enable designers an easier path to exploring physical computing. They have built a programming library called Beholder, which empowers users to develop their own software leveraging computer vision (CV) markers. They presented this research at CHI.

Marking Material Interactions with Computer Vision
Peter Gyory (ATLAS ACME Lab member, PhD candidate), S. Sandra Bae (ATLAS ACME Lab member, PhD student), Ruhan Yang (ATLAS ACME Lab member, PhD student), Ellen Yi-Luen Do (ATLAS ACME Lab Director, Computer Science faculty member), Clement Zheng (PhD alumnus, ATLAS ACME Lab)

[video:https://youtu.be/xYf1VJoqpBQ]

Do notes that ACME Lab has focused much energy on developing platforms upon which others can build out their own ideas. Put simply, she says, “We celebrate building things to help people build things,” including useful computational tools, process documents and 3D-printed devices. It is not just about prototyping; instead, ACME Lab aims toward “iso-typing”—functional tools that empower creatives, artists and scientists to make new and better things more easily.

With the ACME Lab’s simple toolkit, designers no longer need to study advanced computer vision to use the technology. Instead of spending too much time just figuring out how to get things to work, they can focus on rapid prototyping with alternative controllers and interfaces to build their own gadgets much more quickly. Do and her team presented this concept to fellow members of the research community and global business leaders at the Beyond Prototyping Boards workshop at CHI.

It is by putting tech integrations together first and then exploring the design space that you realize more opportunities for innovation. As Do notes, “ATLAS is doing such a diverse research, which is pretty unique,” and creates more opportunities to cross-pollinate ideas and support one another’s work across the institute.

Related research presented at CHI 2023

Fabricating Paper Circuits with Subtractive Processing
Ruhan Yang (ATLAS ACME Lab member, PhD student), Krithik Ranjan (ATLAS ACME Lab member, PhD student), Ellen Yi-Luen Do (ATLAS ACME Lab Director, Computer Science faculty member)

[video:https://youtu.be/v9W9n_Lstns]

Facilitating Physical Computing with Computer Vision Markers
Clement Zheng (PhD alumnus, ATLAS ACME Lab member), Peter Gyory (ATLAS ACME Lab member, PhD Candidate), Ellen Yi-Luen Do (ATLAS ACME Lab Director and Computer Science faculty member)

[video:https://youtu.be/c_pXZIerZY0]

Off

Traditional

White