Interplaying – Interactive Media Installation for Live Music Events (Thesis, Vassar ’16)

Updated 2018/1/21
*Finally uploaded the pop-up showcase video:

**You can also check my thesis paper here:


Here, I am sharing everything about my thesis project, Interplaying—from my inspiration/motivation to work process/presentation. Thanks to everyone who supported me and I would be glad to see this project being useful to any hacker/maker projects.

Project Summary

 In the 21st century, there is a strong trend of the audience’s personal technology-dependent behavior in live music events, specifically music concerts and music festivals. This project investigates the way to encourage the audience’s human-to-human interaction to allow them to better experience one of the top benefits of live music events—socialization.

인터플레잉은 라이브 공연에서 관객들이 핸드폰이나 카메라와 같은 개인 디지털 기기에만 집착하는 현상을 타파하고 서로 다른 관객들 그리고 아티스트의 공연과 더욱 그 순간을 즐길 수 있는 경험을 선사하기 위한 프로젝트입니다. 아래의 슬라이드는 논문 발표 시에 쓰였던 슬라이드로써 핵심 주장들을 간단하게 요약하였습니다.

이 논문 프로젝트를 시작하면서 다양한 영감과 모티브를 참여자 간의 자연스러운 교류가 활발하게 이루어지는 지역축제들에서 갖고 왔습니다. 성공하는 지역축제들은 모두가 공유하는 하나의 미디엄, 매체를 가지고 있었습니다. 그 아이디어를 바탕으로 라이브 음악 이벤트/공연/축제에서도 사람들이 직접 가지고 놀 수 있는 매개체가 있어야 한다는 의견을 제시하고 음악과 기술을 이용한 미디어 아트를 만들었습니다.

It found a lesson from organic and physical human-to-human interaction in cultural/community festivals. Also, it does not criticize the presence of technology itself in the music events. Interplaying tests if technology can help and encourage live social interaction amongst audience members, bringing people together in real world. The video of implementation and specific making process of Interplaying will be more uploaded in my YouTube Channel:




I initially started exploring the technology in live music events (music concerts and music festivals). Then, I found these interesting images (#2 and #3 in photo series above) sarcastically commenting on how audience members are tuning into their own personal devices, like digital cameras or cellphone, while being surrounded by hundreds of people around them. (Of course, we would need a drone to support our WiFi-s and maybe a 30-inch LCD screen for better view).

And due to this device-attached behaviors, we are missing something—face-to-face interaction, language and flesh. There are already many concerns about our social capitals negatively influenced by technology in sociology, psychology, and many other fields. Sherry Turkle, a professor of Social Studies of Science and Technology at MIT, wrote multiple books about the relationship between humans and computers/technological devices. Two of her books, Alone Together and Reclaiming Conversation: The Power of Talk in the Digital Age, were my go-to-books during my thesis period. They provided me new critical perspectives and inspiration to think about meaningful interactions for human beings.

While focusing on the interactions in live music events, I also searched local events, regional or cultural festivals. Why? Although they might be completely different—target audience, music style, set up location, etc, they are so full of organic human interaction between the audience members. I decoded what makes this possible in cultural festivals is that they have shared medium.

Coming back to live music events, I thought there should be a reason why people visit public live music events instead of just listening to music at home, live-streaming Spotify or Apple music or Pandora, you know. Similar to how regional/cultural festivals share so many interactions with other participants, socialization is one of the main reasons why people still visit live music events.

Understanding that socialization is an important part of the live music event experience and the history of how shared medium successfully encouraged social interaction in festivals, I created the shared medium (using hand motion detector and gaming program) to generate communal activity that the audiences can play with one another.

There were two big questions that came along with the usage of technology. First, why not get rid of it? Commercially and industrially, it is realistically hard to completely stop using it. Easier said than done. Second, technology does not inherently isolate people. As the quotes imply (3rd to the last photo above), it is how we use the technology. Thus, I created the interactive media installation that would bring people together in real life, initiating eye contacts, short conversations, even small physical contacts.


본 미디어 아트는 DJ들이 쓰는 Launchpad의 형태에서 영감을 받아 버튼과 같은 비쥬얼로 만들었습니다. 손동작 감지 하드웨어 위에 손을 올리면 본인의 손을 화면에서 그래픽으로 볼 수가 있습니다. 화면 속의 본인 손을 움직여 큐브를 건드리면 큐브 안에 삽입한 음악 파일 (filler)가 재생됩니다. 본 데모는 뮤지션이 라이브 공연을 진행하고 있다는 가정 하에, 흘러나오고 있는 음악에서 직접 쓰이는 하모니 및 필러 (3~5초)를 각 버튼에 삽입했습니다. 버튼이 재생될 때마다 노란색으로 바뀌어 본 버튼이 재생되고 있음을 알리고 그 사운드 파일이 끝나면 다시 빨간색 (본래 색깔)으로 돌아옵니다. 이 미디어 설치 아트는 사람 간의 인터렉션을 증폭시키기 위한 것이 목적인만큼 여러 명이 함께 사용할 수 있는 것이 가장 이상적입니다. 다만 이번 실험에서는 하드웨어가 4개의 손 이상을 감지할 수 없는 관계로 2~3명이 최대였습니다. 그래도 서로 다른 사람들이 (모르는 사이여도) 이런 매체를 통해 같이 놀 수 있는 환경이 제공되어 대화나 눈빛 교환등 인간적인 교류가 오고 갔습니다.

How Does It Work

Put your hands on top of Leap Motion device
Try to touch each red cube, which is similar to DJ LaunchPad button
Each cube has a stem file extracted from the music going on in the live music events
Play it with other people around you
Feel free to collaborate and create the sound together
Feel free to share eye contact, physical interaction, smile with your co-players


Special Thanks to Vassar College Media Studies department

& TimeWaveFest folks & Unity & Leap Motion


Please Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Google+ photo

You are commenting using your Google+ account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )


Connecting to %s