In this episode of Tech in the Arts, Dr. Stephen Neely, associate professor at Carnegie Mellon University's School of Music, talks with the Sofia Akhmanaeva, AMT Lab’s Social Media and Marketing Coordinator. They discuss eurhythmics, a century-old practice focusing on the bodily engagement of music, and how these principles extend to modern interaction design. The conversation delves into the evolution of design practices and the need for a more holistic, user-centric approach in the digital age.
Dr. Stephen Neely is an associate professor of Dalcroze Eurhythmics and Dalcroze license at Carnegie Mellon University School of Music, who also presents hands-on workshops in the US and around the globe focusing on the overlaps between music, design, body aesthetics, performance, and experience.
Show Notes
Dr. Neely’s dissertation “Soma Literate Design: Recentering the Interstitiality of Experience”
Transcript
Sofia Akhmanaeva
Welcome to Tech in the Arts, the podcast of the Arts Management and Technology Lab at Carnegie Mellon University, or AMT Lab. I'm Sophia Akhmanaeva, marketing manager at AMT Lab. Today our guest is Dr. Stephen Neely. Stephen Neely is an associate professor of Dalcroze Eurythmics and Dalcroze license at Carnegie Mellon University School of Music, who also presents hands-on workshops in the US and around the globe focusing on the overlaps between music, design, body aesthetics, performance, and experience.
Hello, Professor Neely. It's very nice to talk to you today. For those who don't know, I would like you to explain briefly what Eurythmics is and what is important for musicians and probably other artists, too.
Stephen Neely
Right on. Well, it's nice to be here. Thank you for the invitation. Eurythmics is a 120-year-old practice that came out of a music conservatory in Geneva, Switzerland, beginning of the last century, the man who came up with the original ideas, his name was Émile Jaques-Dalcroze. And he was working with young performers and young performers who had passed all kinds of auditions and were very fancy on their individual instruments. But, as he met them at the conservatory, it occurred to him that in many cases that were big deficiencies or gaps would be a better way of saying it.
There are big gaps in their prior education where certain aspects of their music-making had been emphasized, and other parts of music music-making musical experience had been deemphasized. He thought, “Oh, I wonder if we can wonder if we can address this in some way?”
So he began, what was his life's work. A life's worth of what he called “experiments,” which were just doing new and weird things in the classroom to see if he could get his students to pay more attention to the “feeling” part of music rather than only the “articulating” or the “sounding” part of music. And, what he ended up with after many years of messing around was a bunch of things, but, maybe the most famous of his work is what we call the Eurythmics classroom, or in Europe, they call it eurhythmique.
And in this classroom, traditionally, there's a big open floor and there's a hopefully a big acoustic piano. Someone who's teaching leads from the piano improvising musical experiences. And the students are led through these experiences in all sorts of bodied ways. The main sort of center of the work is that the role of music is to feel it. More importantly than that, it makes sounds is that it gets in the experiencing body of both the performer and hopefully the audience. And so, by inviting my students to participate in a very kind of visceral, literal bodied sort of way I can ensure that at least for the extent of the exercise they're feeling things.
And then we tie those feelings to musical gestures, little by little. Students who maybe didn't all come in with all of those instincts to begin with, little by little, they start to, when hearing music or when imagining music, they start to develop the sensitivity to feel what that music is doing in their body or with their body.
A big theme, which is maybe something I wouldn't be surprised if we end up talking about, is the difference between what is “embodied experience” and what is “disembodied experience.” And the Eurhythmics class is centrally concerned with that dichotomy.
Sofia Akhmanaeva
Could you elaborate on this, then? Because my next question is gonna be about how technologists affect producing “embodied experience” and “truthful experience,” “artful experience,” you say. Could you then elaborate on embodied versus disembodied?
Stephen Neely
Yeah. And music's a great metaphor. In the Eurythmics class, it's not just a metaphor, it is the central course of study, but it's actually a much larger set of ideas than is only applicable in music.
What we are learning about with music as our main focus is equally and it's easily generalizable to any kind of experience. So if we think, “oh, I'm involved in a thing, or I'm doing a thing, I'm thinking about a thing I'm participating in something.” All of those would be different versions of experience and anything like that.
Whether there's music playing in the room that I'm sitting in, or I'm going to the play where there's a stage actor and the lights are dimmed, or I'm gonna open up my app and swipe left and swipe right, or I'm going to carry around this internet-of-things widget that's gonna give you my bio-data to the central database, whatever the experiment is.
If it's claiming to be an experience that is something I'm involved in, then the only way that I'm actually involved in it is that it somehow enters inside of me. It somehow wraps up me - my own personal motion, my own personal gesture. “My very living” - one of the ways that I word it sometimes - is intertwined or entrained to whatever the experience would be. If we'd ever meet that threshold, if we fail to meet that threshold, then there may be something in the room with me, but it's not actually part of me. And in that respect, it's disembodied from me. So the music that's playing in the room that I don't actually feel, I hear it, I'm aware that there's noises coming from that speaker on the other side of the room, or maybe that person is sawing away on their violin and making noises.
Even though almost anybody, certainly in a Western context, would be able to say, oh, that's the sound of music, it doesn't actually count in my work unless the music transcends the sound on the outside of me, tips to the inside of me, and actually moves me. So causes me to feel inner gesture, which is all different versions of tension and release, or of trajectory or of suspension, and then resolution. All sorts of, there's a long list of ways that we can feel.
But, if my own personal experience hasn't been affected in some way, then even though we all could agree there's music in the room, it isn't a musical experience for me. So we can take that metaphor, then, and then expand it into anything else. It doesn't have to be music. It could be, again, any of the standard performing arts, but we can also start to now look at technology, so much modern technology as mini events that it's inviting me to participate in.
Another example is the swipe left, swipe right - which fortunately I can tell you I have almost no experience with. I have none of this, what I assume that's generic language for dating apps. I don't know if you swipe left and right on other things that aren't dating. But I've been married for 32 years, long before even the internet was involved, let alone these things. But we think, well, what's the goal of that app? And the romantic in me says, “oh, it's to find my partner.”
Like, the best version of those apps is to help me find my life partner, right? That's the goal. But the way I'm going to find my life partner has no bodied buy-in. The literal experience of the tip of my finger touching a piece of glass and then swiping, two inches at most is such a severely limited interaction that I can't even figuratively or literally feel the weight of the choices I'm making.
If you compare that to what it would mean to be at a social event and I look across the room at somebody's eye and their eye looks back at mine, just the eye contact carries weight and it like, maybe it freaks me out or maybe makes me very excited or, but I'm like I'm in it now.
It has moved me in some way. Just the eye contact is a gesture. And that to be pictured in a very literal way, it's emotion that I felt. I had nothing, and then all of a sudden I made the eye contact and I could feel, and whether you're thinking about literal hearts beating or perspiration or just the sensation that something is starting I've initiated the beginning of something, and how will it resolve?
Sofia Akhmanaeva
Let's continue then with a broader context beyond music. You wrote in your dissertation, some literate design, which you also can explain briefly what it means, that interaction design is a performing art, but at the same time you pointed out that it's not quite there to deliver truthful experience that would be a comparable to the analog. Meanwhile, we have all these developments in the arts and like different forms and spheres of arts. When people experiment with VR or AR and it's a new experience, it might feel very true, but let's talk about why it's not completely true in your opinion.
Stephen Neely
So, I think that interaction design, that is the kind of what we think of as proper interaction design, which assumes some kind of technology most of the time certainly can be embodied and it can be authentic and truthful. It's just that quite often it isn't. And a lot of the dissertation was just trying to point out to what I think of as still a rather naive audience or a unrehearsed audience of designers that the body actually matters. It's still weird and funny to a lot of designers.
Like I don't think I'd have too many people argue with me these days, but I also don't believe there's a lot, I don't believe that the body, upfront and central, is a is a required part of everybody's curriculum when they're trying to teach “What is design?” and “What is interaction design?” I'm not sure that the feeling body is at the top of everybody's list.
I hope it's more and more the case, and I'm sure it is more and more the case, but the tradition of design and where it came from so much of what is modern interaction design is still being taught out of a base curriculum, a foundational curriculum that was two dimensional and three dimensional, as opposed to finding ways to comment on the 4D time-based body swept-upness of the things we make. So Soma literate design and that dissertation was just trying to say “hey, you're getting into a new space here.”
I'm old enough and I'm married to a designer and my middle son is a designer, so I feel like I'm deep enough in that world to remember when there was no 4D design. There was no temporal design. No body, like design-proper. The design fields of architecture, communication design and product design didn't really have much need or opportunity to think about the evolving design. The participatory design. It was the designers, of course, with amazing instinct, which is a lot of them, they always had an instinct for it. But there's, there was no practice, there was no sort of field.
As interaction design came along, it was very much technology-driven. Oh, check it out. We got internet, now it does stuff. Let's play with that and check it out. We've got internet of things. It does things. Let's, yeah, let's play with that. And so very technology-driven and not particularly user-centric in any way. We're trying to notice why is it that I wanna fall in love with my phone and can't put it down. And like everybody's been trying to figure that out.
So when I wrote some literate design, I was just trying to introduce, really, a musicianship of design to the design worlds to say - the same model that I described as the eurythmic kind of attention, we can bring that exact same attention to, into any kind of experience, not only the main stage music or drama or dance experience, but also we can look for those same ideas of gesture and gait and inner motion and embodied and disembodied.
We can use that same language when trying to critique, does this app do what I want it to do, or does this service design with my healthcare company do what I want it to do? And then when we think it doesn't do what I want it to do, the hope is that the Soma literate designer would go, “oh yeah, where's the body in that?”
Is it possible we kicked the body out while we, the feeling body out, while we pretended that we were involving them by only giving them the tip of a finger on glass or pretended that we were involving them, because we have their name in our database and they get an email every three weeks from us.
And which I'm trying hard to say, yes, you're, you sent them an email every three weeks, but is their own personal experience feeling anything? Are they involved in a suspension or a emotion or a trajectory that they can, that they feel like they're getting somewhere? So, I'm not sure I answered your question.
Sofia Akhmanaeva
Could you explain why interaction design is performing art? What is the parallel?
Stephen Neely
Well, it should be a performing art, is what I wanted to say. The interaction design should be or aspires to, or is ideally a performing art in that when I am performing and whether we could say I'm the performer or I'm the listener of the performance, or I'm the audience of the performance. You could describe this either way. If I'm the performer on the stage and I have a violin in my hand and I'm sawing away at those strings, that actually doesn't yet assure that I am in an embodied experience. So even performers aren't guaranteed performing artists.
They are, in the ideal way, in order for it to happen, the notes that I'm playing have to somehow speak to and from inside me. So I'm trying to like, make a point, I'm trying to resolve attention. I'm trying to build a suspense, I'm trying to create a cadence, all of those feeling things, which is different than I wiggle my arm on the outside of my body and make it touch strings or touch keys or what have you.
The goal is that it be the performing art. When interaction design is doing the shallow job, it just makes me push buttons. It makes me push buttons, it makes me enter data. It's superficial. It's thin and I go through the steps. If it's really doing the job that we would like it to do, then it somehow wraps me up.
It pulls me in it, it encourages me to feel something and hopefully, with a kind of a goal that it's, I'm feeling myself go toward. So if Duolingo is doing a good job, it's at least partly getting me to show up every day. And so I have some sort of routine where it's become part of who I am.
It's something I do. And so it's at least if it's getting me to show up every day and play the game another time, that some part of that is working. I would say similar to I'm wrapped up in an event. I'm wrapped up in a culture I'm wrapped up in. I feel progress if it had kicked me out of the experience and all that it did was just make me wiggle my finger.
I just have the equivalent of tapping my finger all day long and I can't imagine anybody would do that very long. And there's nothing inner rewarding about that. But if it can get you to keep showing up, there's something going there. I'm not sure it's the best example or that it's meeting all the needs but it at least is getting the person involved in a way that pulls it forward.
And then my angle on all of this is that the involvement is something deeper than just me thinking, “oh, I gotta do this again.” It's meeting some sort of inner need.
Sofia Akhmanaeva
Let's get back to classical performing arts, so to say. And in this interaction between artists and the audience, there are two parts, right? It's artists who should have their own experience.
That is true to deliver artful things to people, but it's also the audience who should be in a particular set of mind to receive it, and then some interaction actually happens. Do you think that the prevalence of technologies in our daily life affected somehow? And if yes, then how? The way both in interaction directions work. Are people less or more like how
Stephen Neely
Well, I don't have the data, so I'm not that person, but I'm sure some people do have some data. Yeah. Certainly, the prevailing opinions is that technology is encouraging a more and more disembodied experience. It's doing less and less to require us to. To to be bodily invested.
It's offering more and more opportunities, and in many cases, nudges to push us out of the embodied experience. Again, you go to the social event and meet your new partner. Or no. That's nobody's instinct anymore. Everybody's instinct is to pull out the piece of glass and swipe your finger on it, so there was no option not that long ago, except you must go to the dance, you must go to the church, you must go to the university or the whatever where people are, and that's where you're gonna find your partner.
And now again I'm assuming that a lot of 20-somethings assumption. Is that no. That's not at all where you start. You start on this app and you start by making your tip of your finger swipe glass. And then there's lots of other examples of that. And I hope I don't sound like a Luddite who doesn't think there's any good in technology.
I don't think that at all. I'm very excited for my international Zoom calls and the ways that I get a chance to interact with people that I wouldn't have been able to interact at all with. And a whole bunch of that is quite embodied. But the fact that that I have this kind of frictionless like it took almost no effort at all to talk to my friend in Chicago just now. I was on a call. Whereas if I was really gonna get the deep interaction with him, I'd have to buy the plane ticket and fly there. And then like, how much richer would that be to be in person than it is to just have this sort of thin interaction. So I think there are all sorts of cases. Again, the Luddite rant would be, nobody reads books anymore and nobody knows how to use a library and, and like we can just think of all the things that are now so easy to do and take no effort.
And on the one hand it's amazing and brilliant and great to have so much access to so many things, but I do think there is some cost. Please don't let me sound like I don't, that I never think the cost is worth it and, in many cases, the cost it we're yielding other things that are also quite valuable.
And I was able to write a dissertation being able to scour all of Google Scholar any second of the day. I didn't have to leave my house, and I had access to so many volumes that I would've literally never been able to get to because the actual print copy, there's one of them, and it's in France, so I would've never been able to do the scholarship that I was able to do as all of us are now.
So there's so many good things about technology. However, there is this bracketed space that has to do with embodied and disembodied. And we look at loneliness epidemic. We look at mental health crises. There's all sorts of bits where the technology claims to be giving us a, an experience, but actually it's the facade or the shallow, or the, it's the facade of the experience, rather it's the inauthentic experience rather than something that would've just been easily meaningful.
And we've done a terrible job. And by “we,” nearly universally everybody, whether it's the coder, it's the designer, it's the teacher, definitely the culture has done a terrible job of noticing what is the difference between something that actually feels honest and true and something that just is the thinnest, shallowest facade of something that is true.
Again, back to the swiping left. So I think when we look at the bracketed space of who am I in my world? What is my agency? What do I have control over? What's it feel like to be alive? And how does technology encourage that? I think a lot of technology is actually discouraging my aliveness, my sort of participation in my world.
I'm not sure that I'm yearning for the days where I had to plant my own garden to get my, all my food. But if I did, I know what I'm doing and I'm physically doing the labor to yield the thing. And I had the time. I couldn't rush it. I couldn't just go out. I had to plan. And the arc, the suspense, the tension that is built up as I'm waiting for my zucchini to finally ripen and then, oh my goodness, it's the day I get to harvest my zucchini.
That is that gesture of my arm above my head. And how long can I sustain it before it's gonna have to come back down and rest? And so those practices of years gone by. The embodied and the personal and the motion filled, and the visceral. It wasn't optional and it was basically in everything, you couldn't do anything without doing it.
And now we can do so many things and really not do it. I just, you know, or certainly not know that I did it. Did I do that or did the thing do that? And what's the line between? So there's something there, there's something there. And I can go back to 120 years ago when Jaques-Dalcroze found his first music students - even then, long, of course, before all this technology, they could saw their bow over top of violin, and yet not, or even easier, wiggle your fingers on top of a keyboard and yet not have it come from a deep place inside you.
And so the basis of his research, which is the basis of my research is there is a deep place inside you. It does feel like living. And if we can infiltrate our living life with those gestures, notice them, search them, design them into experience. Maybe we have a chance of making our daily lives richer.
Jinghong Gong
From the point you were bringing just now, I really love the analogy you were talking about music and design are related to each other. The last point you talked about how modern technology is making this bodily engagement more difficult. Then going to music playing while you have those body engagement movements, it doesn't necessarily mean you have that, I would say perhaps is it emotional connection? I want to say if there’s, a relationship between the embodiment and the intensity of emotional connection or maybe even the duration of how, the impact of that emotional connection. Where is the relation or distinction between the two?
Stephen Neely
Yeah, I think all of that's certainly wrapped up in it. Emotion certainly is desirable and part of it, but it's actually, I think, a higher-level thinking than is even necessary. So as emotion, like my happy, my sad, my excitement, my boredom, or whatever other categories of emotion you'd like to put on anything, those are all great. Those are all wonderful things. Totally useful. There's even a layer below that though, before I've had a chance to even decide “Do I like it or don't like it?” which is just that I was in, I know I was involved.
So before I even have a chance to make a choice, to have an opinion, I first have to just participate, and then my opinion comes second. So the emotion that would come from something is a higher level analysis than is required to just participate. And so to know that I was in it, like, I could come up from the side of you and you didn't quite see me, and I nudge you on your shoulder.I'll just give you a little shove, nothing violent, just as the littlest bit, just and then you don't know yet to be offended, or excited because you don't know where it came from. All you know is that something happened, there was an event, and it moved you, it touched you. And in this case, very literally, so it, it moved you. It touched you.
And so then you have a moment of reflection and you turn back on it and you say, oh my gosh, it was my best friend. I haven't seen you in forever. Or you think, oh, it's a jerk. He's always bumping into me. He doesn't know. And so then you have an emotion. But the emotion, good or bad actually wasn't a requirement for you to have just participated, for you to have been in.
So if I understood your call, so what I'm mostly interested in is just let's just have the participation. Let's notice when we are involved. And that same thing could have happened. You're on a balcony at a shopping mall and you look at the crowd below and you think for a half a second, “oh, that's my best friend from years gone by.” Just a half second. It didn't actually bump you literally on your shoulder, but you think you saw that person, and so for a moment you have this suspense of, “oh, what's going on?” and this slight excitement. Then, you have to decide what's your emotion about it. I know that person. I don't know that person. They're real. They're not real. But, you had the experience. It's in a, maybe a very similar way to me, shoving you in your shoulder. Something happened just now. It wrapped me up. It touched something on the inside of me. It's unresolved until it resolved. And so all of that is what I would call a gesture, some kind of inner experience, inner motion. The shoving of you is an external actual thing that happened, but the meaning behind it the gesture that we're trying to get to. So I like the, although I think it's a deep example of musicians wiggling their fingers on piano keys.
So your listeners will be patient with me as I try to say it, but we all know that we can wiggle our fingers in lots of ways. It doesn't make it a meaningful thing. And some of us have learned to wiggle our fingers in ways that make culturally desirable sounds. “Oh, look at that. That was a consonant, that was a major chord. And oh, it's a blue scale. How exciting, that person knows a blue scale.” And yet, that doesn't mean that the person playing it was using it as a way to translate something of their inner experience. The best artists or the artists that I want to work with, or the artists I'd like to raise up, we start with “what do you want?” And then we try to use our instrument to share that with others. And I think if we then try to flip it back to technology and designers and tech and the AR, VR, or the, even your little apps on your phone, we could certainly say, well, what is the experience you want?
What are you trying to help this person do? It doesn't have to be profound. It doesn't have to be as big a deal as a full stage opera or life changing as finding your life partner. It really could just be no. I just want them to smile for a minute playing Candy Crush. Or I want them to, like it could be something very silly or dumb or simple or short.
But if we, as Soma literate designers, can just start to notice the difference between personally meaningful and absolutely shallow or inauthentic or fake or facade or just going through the motions of something, then we at least could more clearly talk about what we're doing. I know I'm designing a throwaway interaction. I don't expect it to be anything more than this. I only need them to do it for freaking six seconds, and I get everything that I need it to be. I'm never gonna see this person again, and it's probably perfectly fine. But if what I need is a relationship, or I want the person to feel like they're a member of a club or a member of a community, or what I need is as a much more long-term commitment, and I need them to still be a member of the study 20 years from now.
So I have to get something going that they feel like they're a part of it. And that is all kind of inner work. There's no on the surface of me work that will be able to sustain something that deep.
Sofia Akhmanaeva
We talked about this line between true and not true and it's, to me it feels like, it's like vagus nerve. It's very vagus line. It doesn't lie somewhere as an actual line that doesn't have any curse and just go from point A to point B. I will give you two examples and I want you to say if it's true or not and why? One of them, the first one is more general. About using generative AI in visual arts? When people use AI to create part of the work, is it true? Is it not true? Does it make work real or not? In which particular cases.
Stephen Neely
I think the answer goes back to like “true for who” is a really important clarification. So I don't think anything just is. It's always “is for,” or “on behalf of who” like, you used the example a little bit ago of the, there's somebody on stage making music and there's somebody in the audience making music.
Is the music true? You didn't say that part. No, I'll add that. Is the music true? Well, actually it's, that's an incomplete question. Is the performer coming from, some sort of personal position, is it true for them? That is to say, are they sharing something authentic? Is it, does it feel like motion to them?Does it come from something deeper inside them than they're trying to relay it? That doesn't guarantee what the audience member is going to get.
We think as music professors and as main stage performers, we think we have a decent amount of control over it. But actually we have very little control over what an audience receives. The audience member can. The question is not was the music true? The question is, did they have an authentic experience inside themself and they can have an experience based on great playing by my standard or horrible playing by my standard. By live in the moment, acoustic, literal vibration, analog playing, which is everything I love. They also could have that by an AI-generated, algorithm and it somehow swept them up. It somehow convinced them that something happened inside them. And I think it's very possible that all kinds of AI-generated stuff in our future will be impactful in all kinds of ways.
I'm hopeful, and I'm pretty sure it won't be impactful in all ways or in all the same kinds of ways that an analog live human is going to be able to pull off. But, I'm sure as the technology gets better and better, we will all be fooled in more and more ways and to, into believing.
And I'm not even sure that it matters. Like for the rant that we're on today in this interview, it doesn't matter that it be live. That isn't the only thing that is true. Because what's true, the truth that we're looking for is the truth inside of you. Did you have an experience? Did you feel something? Now if what I would like is a relationship, then we could go off on another whole thing about the way we relate to our AIs, and can you fall in love with your AI and should you be dating your AI, which I gather now is totally a thing. And there's a whole other pile of whoop ass that's about to come out with all of that.
But I'm not sure that it changes what is authentic to the individual. That is, I can be lulled into all kinds of environments and feel like it was meaningful to me. And which is maybe a lot of the danger, like one of the big existential dangers of the AI that's coming is that it could lull you into something making you believe or have a series of experiences that then it will have to fall short of it some further way compared to the human.
I'm still working all this out myself. I don't know all these answers, but I do think, at least for the first part of your question, AI art, AI-generated art can be beautiful to the person who's looking at it, and it can just, as all kinds of human-made art is horrible to lots of other people, they're just like, I would never engage with that.
Sofia Akhmanaeva
I'm glad you brought it to the audience side because the next example, and I believe the answer that you just gave is applicable to this example too. But this example is about specifically the artistic side. So in Sweden, researcher and composer Frederick Grant designed and developed a cello-playing robot for another Swedish composer.
And this robot performed with Malmo Symphony Orchestra and the composer, Jacob Mul Murad said that such robot will of course not replace a musician, but this kind of art can become its own art form. So if we have the robot literally on the artistic side, how does that affect the truthfulness of experience where it's still on the audience side?
Stephen Neely
Well, it's truthfulness of experience, again, it's not just one thing. So there is an audience side, and I think the audience will be wrapped up in whatever they're in the mood to be wrapped up in or whatever they allow themselves to be wrapped up in.
And the instigator of the wrapping up of the getting swept up either the live cellist or the robot cellist, or the algorithmic just audio cellist that came outta the AI. They'll be more and less successful in different cases. But it's really that audience member who is open to an experience and has found something in it that feels like something to them that they're willing to get swept up in.
On the part of the creator though I am quite happy with my definition that says artistry begins with choices. And so if the AI isn't actually making choices, if it is making choices or it isn't making choices, I'm pretty sure that AI doesn't make choices. I think it is grabbing from some sort of data pool and ranking things.
But I, I don't think it's actually in the choicing level. So I'm pretty sure that I'm confident in saying I don't believe any AI can be an artist, which is not the same thing as it can't create an artful experience or can't be the instigator of an artful experience. But I don't think that an AI can be an artist, even if it makes an image that I've never seen and that somebody thinks is beautiful.
Or if it can create a musical line that we've heard or never heard before and somebody thinks, oh, that was moving. I think the move of the artist, which at this point, I'm pretty sure has to be a human flicking in my head. Can dogs be artists or can monkeys be artists? But I don't think they can, I don't think I'm ready to say that.
Is that it's this kind of very special exercise where I'm playing within some kind of bounds. I have a boundary, a sort of a set of assumptions, and then I'm pushing on those assumptions in some way to see where it takes me. And so the choice part is, oh, I know that's an edge. I'll push on the edge and we'll see what it, does it break? Does it expand? Does it stretch? Does it squish? Does it reveal something new? Does it make noises I didn't expect? Like all of those kinds of things, that's, that all falls under the heading of artistry for me. So I spend a lot of time with my musicians who are mostly, 20 year olds. So they have a nice amount of life experience but I'm pushing triple their age these days, so it's still quite limited. And I'm always on the lookout expecting that many of my students are just going through the motions. That's just what I expect of a good 20-year-old is that they, a good musician, 20-year-old, is that they're not quite in the artistry game yet.
They're not quite making choices. Certainly they are in some cases, and here and there, and certainly some students are amazing. When they were 14, like there's lots of that. But on the average. A younger musician is still just trying to figure out how to make their instrument work.
They're just trying to make it, make the sounds they want it to make. And so rather than creating things, they're just trying to just make it make the sounds. And the easiest way to do that is just to do what somebody told you to do. I'm following this score, I'm gonna play these notes. Or I'm, I'm playing the notes that somebody wrote on a piece of paper, or I'm playing back the melody that somebody else made and I'm trying to play it like they play it.
I love the way they play it. If only I could make it sound like that person. And so they're trying to, it's a kind of, rudely, I'll say paint by numbers. Like just somebody, just give me the recipe so that I could do it. And you can get far with that. It's how everybody starts.
It's how I start. It's, I still have all kinds of parts of my life now where I'm painting by numbers because I'm just not good enough to be artful at the thing I want to do. I think there's almost no other way to do it, except that you have to mimic and imitate and just let somebody tell you what to do.
I'm trying to bake bread right now. I am not, oh, I made some pretty bread last weekend. But I'm nowhere near an artist. I am just barely holding on to get my sourdough started or to do what it's supposed to do. And so, what we're looking for in the students is that the opportunities for them to offer something of themself for them to stretch the practice for them, stretch their practice.
Maybe it would get to a point where it would stretch the practice of violin playing or of, orchestra composition or of bread baking. And I'm not sure that AI can do any of that stuff. In fact, I'm pretty sure my understanding currently is it absolutely can't. It can only offer what is in some version of the data set it already is in possession of, so I'm not sure it can innovate anything new. I'm not sure. You guys maybe know better than me, but I, if I understand it a little bit, I. The AI can only work with what it's been given and it can draw things that we didn't realize was a pattern. It can reveal things that were already there, but I think the point is they were already there.
It's not actually new. And whereas the artist aspires, as does the academic by that by the way is to create new knowledge to push into a space that we've not like really gone there before. , and every once in a while somebody hits gold. But mostly we're all just stumbling along trying to push on our envelopes.
Sofia Akhmanaeva
Could you talk more about your current experiments and research in motion about motion and technologies?
Stephen Neely
Let's think.
Sofia Akhmanaeva
You mentioned that you do something with motion capture tech to produce some kind of artwork.
Stephen Neely
So I've played in those spaces a bit in the past. I'm not doing any of that kind of stuff right now.
Sofia Akhmanaeva
You mentioned Arduino and making wire sculptures
Stephen Neely
Yeah. Okay. I can talk about that.
What I'm working on now, my current research is trying to, it's basically a dissertation part two. It's an extension of the ideas I began 10 years ago where we're now trying to lay out a framework for practicing experience designers. So whether they're experience designers, service designers, any body interaction designers, anybody working with body and time in their designs. And so it's all versions of service. It's all versions of experience and lots of things hit that threshold. Even architecture which we think of as just like bricks and, buildings, has deep experiential components to it, which architecture, of course, has been more and more aware of the last 25 years.
Anyway, part of what's going on there is a describing. We start by saying, this is what a body is. This is what the experiencing body is all about. Here's a body, it's flesh, it's inner feelings, it's emotions, it's all the things that bodies are and it's beating, it's living.
It's never static. It's always in motion. And so we start with there is this thing called a body. And then there's the second chapter is all about this is what time is. And by time we mean experienced time. So there's kronos time, which is stopwatch time, the “tick tick” of your watch kind of time, which is a life-depleting kind of time. And there's kairos time, which is we call it the opportune moment kind of time where things come together. It's serendipity, it's a vibrance, it's aliveness. The kind of time that wakes you up. The feeling of time of the feeling of time mounting, not the feeling of time depleting.
So we spend, chapter one is body and chapter two is time. And chapter three is all kinds of variables of experience that, that a designer could mess with, like meter and tempo and agogics and rhythm and, orchestration or, the ideas of like togetherness versus separateness. So there's a whole bunch of discreet concepts that we can critique any experience with a lot of very musical concepts in there.
And then we set up a framework. That says, okay, some stuff here are some steps to go about making or analyzing any experience or any temporal or time and body-based design. And I don't know if I have these memorized. Step one is reveal the in-time experience. So when we say what just happened there. I went to the bar and I made eye contact with the person and then they shut me down. And then I went home feeling defeated. So the question would be, not “what happened?” but “what's the soma literate read of what just happened?” And so the common way we would describe it is a shopping list. We would say, well, thing one that happened is I got in my car and I drove to the bar. And then thing two was I opened the door and I sat at a table. And thing three was I ordered a drink. And then thing four, the fourth thing that happened, was I saw the person ,and then thing five was they scowled at me and I went home defeated. And so we like a shopping list. Ding ding. There's five things that just happened. None of that is the experience.
The actual experience of experience was all the space between those shopping list checkpoints. So those were checkpoints. They happened, they were markers, they were what's the word that means, like flags on the road. They were little flashes of something happened, but the actual experience was the space going toward and away from the thing that I made my checkbox about.
And so the experience part of making the eye contact wasn't the instant with that, our eyes locked. It was the build up. To the instant that our eyes locked. And then once our eyes locked. That happened in a flash, not even measurable how long that took. The instant that our eyes locked, the instant that our eyes locked is the edge of a knife.
There's no way to hold onto the instant that they actually locked. What I was left with was all the space that our eyes remained locked, and what was the feeling of feeling insecure as our eyes looked at each other. And so the experience of that one, part of being at the bar was the build up to toward and away from the eye lock.
And so the first part of this framework is to say you have to, you gotta get in that space, the space between the shopping list, check marks, and then. Once you are in that space. The second part is to look for the gestures. So first, I realize I'm not looking for a bunch of post-its on a wall.
They're just gonna tell me all the things that happened. I'm looking for I'm looking for space between Post-it notes space. Between check boxes. Then I'm gonna describe what those are. What were the gestures of the event? And so what was the bodied motion? The inner heave and flow. The tension and release.
What, how many motions were there in the thing I'd like to describe? And if it's a really good experience, they were continuous and one led to another. And on all of a sudden, a thing happened. Like all of that together happened. And then the third part, which is what, took me a long way to get to this, is what were the shape of those gestures?
What's the shape of a gesture? And the thing is not everything feels steady or single-directional. Sometimes things build and mount. They swoop and curve. They use ricochet, they jag, there's all sorts of ways that an experience unfolds.
Some things unfold just so beautifully that it's just like magic. It's like the, it's like beauty in motion. But most of mundane life isn't nearly so beautiful. It's like awkward and stumbly. And so we've been trying to figure out ways to get people, designers who might read this research to understand this very abstract idea of what would be the shape of the gesture hard enough to get people to even notice that their gestures happening all the time.
Your life is a series of gestures. Your life is not. A check, a shopping list of checkboxes, even though that's seems to be the only way anybody knows how to describe it. It's “what did you do yesterday?” “Well, I did this checkbox. That was my day.” No, it wasn't. Your day was none of those things.
Your day was all the space between those things. The way you got to and toward and away from those check boxes and the way you got toward and away from them was more or less beautiful and awkward. Some of them, it just flowed. I got right from where I was to where I wanted to be, and then onto the next thing and just bang.
I couldn't have been simpler or smoother. So what was the shape of that? Well, it was probably some sort of like simple arc, or maybe you thought of it as a straight line or very direct, very clean, very simple. But then I was trying to just, all I wanted to do was sweep the floors of my kitchen, but my dog kept getting in the way and then my kid this, and then the doorbell rang and I just could not get that check box checked off.
And so it was all this zigzagging and starting and stopping and then restarting. And so I had to go backward and forward and backward and forward and eventually I made it through the event. It took these 20 gestures, the gestures, the shape of those gestures were in all of these, not straight line, beautiful arcs.
It was a bunch of kind of weird jaggeds and interruptions and whatever. And so we've been trying to think how could you represent the shape of that interaction. So we were just brainstorming a bunch of ideas. So you could draw it on a piece of paper with pencil and, pencil and paper.
You could you could choreograph it as a dance. You could just, here's my interpretive dance of my day trying to sweep my kitchen floor. Here is there's, there are these CNC wire-bending machines that exist under the sun. It's got a spool of wire and then a bunch of a series of little servo actuators.
And computer knows forward the wire at a certain pace. And then relative to the pace of the wire, which is. And the best version's variable relative to the pace. The actuators are gonna bend the wire in one direction or another. It's how paper clips are made. So the paper clip comes through a machine and the machine knows, “oh yeah, now it's time to bend it and bend it this hard, this fast so that it gets that perfect curve. And then now don't bend it. It has to be straight for a little bit, but now bend it again and it can rotate the wire and bend it in different ways.”
And we thought, oh, this wasn't our completely our idea. There have been lots of artists playing with bent wire for a long time to represent lots of things, but we just thought, oh, that would be a fun way to experiment. I wonder if I could bend a wire to be the shape of the interaction. And so that's where my motion capture questions that you're recalling came from, because there was a couple weeks, a semester ago where I was very wound up with the idea of motion capture and taking the data from that and feeding it into a wire bending machine and see if I could get it to wire bend the choreography of you doing anything.
Then I thought of it further when you would've heard me talking about that I was new in my thoughts on that particular thing and what I realized was I still like the wire bending thing a lot. But what motion capture, the only thing motion capture can capture is the literal motion of me sweeping my kitchen floor or of me doing whatever.
It's actually not the part of the experience I'm interested in. I'm not interested in what did you literally look like? I'm interested in what did you literally feel. So what was the feeling of tension?
What was the feeling of release? What was the feeling of aggravation? What was the feeling of of ease? When was it like the easiest part? And so it's just simple and straight and then what was the part that you could feel it mounting? And so maybe it's got some kind of arc to it. And then what was the spot where you felt stuck?
And so how would we represent stuckness in a bent wire? Maybe it twists on itself or it gets a knot in it somehow before it moves forward. And so it's that kind of inner hidden part of experience that my, that I am dissatisfied with motion capture, because it can't capture the inner.
Sofia Akhmanaeva
It sounds like there should be some other data set representing how brain works
Stephen Neely
Yeah. Yeah. And then we could say, yeah and how much do we wanna call it brain, and how much do we wanna call it inner feel? I like this word called the interoceptive, which is the inner sensation. I think it's helpful to say what I inner feel isn't always what I inner think our goal would be to get the two aligned so that I could think and feel as one.
But quite often our brain trails off and our inners still are feeling like I can say, you could jump out of a corner and scare me, and then I'll scream and then I'll laugh because that was so ridiculous. Or I'll be angry with you because of whatever. But I can't. But I know there's nothing to be afraid of.
But the adrenaline that's already in my body is in my body. And so I'm still feeling anxious, even though my mind tells me there's nothing to be anxious about. And when we're in our most holistic us, everything gets aligned and we get to feel what we think and it all goes together. But yeah, I would love to have that data set.
That is my inner feels, my inner gesture story. That was a lot of words. That was so many words.
Sofia Akhmanaeva
Thank you very much for all your words. It was a pleasure to talk to you. Yeah. I wish we had another hour
Stephen Neely
Well, you can always call me up. These are fun questions, and this is totally the stuff that I'm spending so many hours right now thinking about. So, it's a good exercise for me to try to say things out loud.