Can Robots Feel? With Robert Shepherd and Ilayda Samilgil

Welcome to StarTalk, your place in the universe where science and pop culture collide.

StarTalk begins right now.

This is StarTalk Sports Edition.

We’re going to talk about sensors in this episode.

More on that in just a moment.

First, my co-host, Chuck Nice.

Chuck.

Hey, what’s happening?

Professional stand-up comedian.

You do stand-up clubs, right?

Pretty regularly.

Yeah.

I mean, that’s the only way you can be a comedian.

Also, Gary O’Reilly.

Gary, former soccer pro and soccer commentator.

We’re here, my co-hosts.

All right, Gary, this is one of yours again.

I just know the title of the show, Sensors, but you’re going to have to take us into it and tell us where it’s coming from and where’s it going.

So, start us off.

So, the show begins in space with a suit that has been designed for the planetary exploration of the moon and Mars.

In a galaxy far, far away, a time long, long ago.

Perhaps.

Perhaps.

We meet someone who not only worked on the space suit in question, but who wants to give robots a sense of touch.

Now, this is going to press a button for you, Chuck, but plenty of show to go.

I was about to say, who the hell idea was it to do this show?

This means us getting into soft robotic technology.

We will explore smart garments that are fiber optic based and therefore more photonic than electronic.

That I don’t have a problem with.

That’s more haptic.

Alright.

That’s more haptic augmentation than it is giving a robot a sense of touch.

That’s helping me be able to touch things in a better fashion.

I’m cool with that because it helps me, a human being.

Sorry.

I’m going to try and give you a soft landing here, Chuck.

We’ll see if you get a few bumps along the way.

Right.

We also meet another co-founder who is part of this team that is driving forward this particular project in so many different directions.

The particular technology has multiple medical applications, think computer gaming, wearables.

They’ll go there.

It’s a fashion item.

And yes, you guessed it.

It has sports applications as well.

In particular, it’ll be technique analytics, injury risk reduction, bio data feedback, such as breathing analytics and much, much more.

And you’re going to love this, Chuck.

The inner house husband that you are, it’s also washable.

Oh.

Well, how thoughtful.

Exactly.

Right.

Let me introduce our two guests.

Ilayda is CEO and co-founder of the Organic Robotics Corporation, graduate of Cornell University in mechanical engineering.

We’ll also meet Dr.

Rob Shepherd, associate professor at the Sibley School of Mechanical and Aerospace Engineering at Cornell University, studied at Harvard in George Whiteside’s Department of Chemistry and Chemical Biology.

Where, of all things, he realized he was going into robotics.

And no surprise to see that he is CTO and co-founder of the Organic Robotics Corporation.

Awesome.

Welcome, guys.

You should say your name so that we can all just know it.

Ilayda.

And your last name, please.

Go ahead.

Samilgil.

Samilgil.

So what is the smart suit thing?

Like, what is a smart suit, we all have these images of Neil and Buzz on the moon with these big bulky suits as they’re skipping along, but they don’t look very nimble and they don’t look as comfortable as perhaps they could be.

I don’t know.

So what, so our image of being in space is that, right?

So how are you going to change this?

Well, I think I have a collaborator, Ana Diaz-Arteas at Texas A&M and she makes, she’s into bioastronautics.

She wants to make space suits that astronauts want to wear that can maybe perform more like athletes here on Earth.

And what Buzz had to do was fight against pressure in the suit to bend his arm.

And one of the things she wants to do is make mechanical counter pressure suits, which is more like what you see on the Star Trek movies where it’s conformal stuff and it’s squeezing your skin to keep that pressure differential from, you know, popping you a little bit.

So, I hadn’t really thought about that because I’m in basically zero pressure atmosphere, right?

So, you have to put pressure back on me so that my blood doesn’t boil or weird things don’t happen, right?

And so, with that pressure, I can’t move in my joints, right?

Because there’s pressure everywhere.

So, you need some kind of mechanism to get through all the joint parts, not only my elbow but my shoulder, my fingers.

Yeah.

So, there’s two ways to do it.

One is to use the same suits are being used today and augment the force so that you can get help fighting against that pressure.

Or you can remove the internally pressurized suit all together and just have like a compression garment that squeezes you.

And then your joints are less hindered to move.

Like in a flight suit when you that squeezes your lower extremities so that the blood isn’t draining down and it pushes everything up so you don’t pass out.

Yeah, it’s pretty similar to that.

So a giraffe actually has something like an evolved G suit.

So they have a high pressure from their heart to feed blood to their brains.

But when they lean down to drink water, their heads don’t explode because they have fascia and skin.

That’s good.

I think giraffes 101, their heads kept exploding.

By the way, that is a great name for a bar, the exploding giraffe head.

Come drink it or we’re the whole.

So the more as he can be like giraffe skin, the better.

So Ilayda, what does organic have to do with any of this?

When I think of material science and I think of robotics, biology doesn’t ever enter my head here.

So what is the organic part of this?

Well, I think that’s actually a better question for Rob because we just took his lab’s name and then added corporation instead of lab.

So that’s why we’re called Organic Robotics Corporation instead of Organic Robotics Lab.

But we’re now going for Light Lays, which is the name of the technology, which we can talk later, but you can talk more about the organic stuff.

It’s not food.

Everyone’s like, oh, is it food?

Organic means-

USD is certified.

Yeah, you have organic robots in the organic aisle of whole foods, right?

What’s going to be next?

It’s just organic chemistry, using chemistry to make robots.

And now people are better than machined aluminum and bolts and stuff like that.

So whoever wants to answer this out, the two of you.

Soft robotic technology, what does it actually mean?

And am I right in saying it’s kind of like biomimicry here?

Or am I on the wrong path?

Yeah, I’m glad you gave me an opportunity to do that because I think we’re evolving towards where the company started and, you know, my lab sort of has taken its own direction.

So I started off at Cornell in mechanical engineering, taking what I learned at Harvard, making soft robot machines.

And what they are is just continuum deformation, smooth deformations that don’t look like hinged robots from the, you know, that you think of from the fifties.

So they’re made up of materials like animals, they move more like animals, but everything we made at Harvard was what’s called feed forward in a robot.

Like you issue a command and then it just does it, but it can’t respond to the environment because it doesn’t have any sensing.

So what we did in my lab here at Cornell was to add soft sensors to the system, touch sensors, so it could feel what it’s interacting with and then respond to it and make a truly feedback controlled soft robot.

And so then it can feel and respond and look like an animal.

And we use optical light guides instead of electrical sensors for a variety of reasons.

And Ilayda was in my lab making these sensors for robots and then decided, why don’t we just use these on people instead?

And then that’s when the company started and that’s when the lab continues to make stuff for robots, the company makes stuff for people.

Okay, by the way, that’s the plot of a disaster movie, right?

With a brilliant engineer saying, let’s do this on people.

And everything turns for the worse.

So how do you scientifically go from knowing, as our fingers can touch and sense, to being able to interpret, touch and replicate it for one of your technologies?

It’s very basic.

We cannot do it nearly as well as organisms do.

There’s a variety of mechanoreceptors in our skin that respond to different pressures and different frequencies of pressure, temperature.

And this is a lot of information that we have to encode and then send to our brains.

And we only have a limited space to do that with.

So we encode this multitude of information into electrical signals spiked through our neurons.

And then our brain interprets that information and then sends commands out.

One of the reasons we use optical systems in my group instead of electrical ones, you can encode more information optically than you can electrically.

Hence the urge to lay fiber optics for media communications.

You can pulse at high frequencies, but you also have phase information, color information and intensity.

All of them can represent a different mode of touch feedback.

And you can encode that into one signal and then send that to the computer, which then sends the response.

Using light instead of electricity, does that mean you are immune to the electromagnetic pulse that the aliens will send down and cripple our civilization?

Is that right?

It is a big, it actually is a, you know, it’s a very important point you bring up.

We’re immune to electromagnetic interference.

So you could potentially use these things in like MRIs or other places where magnetic fields can penetrate or electric fields.

So yeah, whether in a cockpit, there’s a lot of places where you might want that.

And be resistant to aliens.

Every time the aliens come, they shut off your electricity, right?

Yeah, with the EMP.

Yeah, everybody knows that.

First line of attack is the EMP.

That’s right.

And then they’re just like, clearly they’re using some type of optic system.

We’ve underestimated them.

Unfortunately, we still have to convert it to electrical signal.

So there is an op…

So how accurate is this fiber optic ability?

So because I’m…

As Neil said, it’s media, it’s TV, it’s a telephone connection, but you’re using it in a totally different way.

So you’re obviously using…

How are you interpreting how that light changes in terms of the results that you then decode and interpret?

We use color and intensity right now.

And we make our sensors so that they can change color when they’re stretched and they can change intensity when they’re stretched.

Stretching can be linear or pressing or something like that, but we’re not as sensitive as something called a fiber-bragg grading optical waveguide, which they’re sensitive to hundreds of nanometers of movement.

We are sensitive to tens of microns of movement.

That’s like a human hair diameter.

But we think that’s sufficient for robots and people.

And by making that trade-off, we can make our systems much cheaper and much smaller.

It would seem overkill if you had nanometer sensitivities.

There’s nothing we do in our life that we’re that big.

I think that was a nanometer off.

No wonder I missed that pool shot.

Yeah, just think, people want to know an enemy is a billionth of a meter.

But a human hair, that’s something we can see that means something to us.

We can feel that.

We decided that was a good target because yes, exactly what we said, you can feel that, you can’t feel a nanometer.

Right, right, of course.

We’re going to take a quick break, but when we come back, more of the secrets of this emergent technology of these spacesuits that will basically replace humans and we’ll all die.

I’m pretty sure.

Is that where this show is going, Gary?

No, I did it because you were happy a note.

Okay, all right, so we’ll be right back.

We’re back, StarTalk Sports Edition.

We’re talking about sort of robotic suits for all manner of applications, not only that we thought of, but maybe stuff we haven’t even dreamt of yet.

We’ve got two mechanical engineers turned roboticists here.

We’ve got Ilayda Samilgil and Rob Shepherd as our guests.

So let me start off.

Ilayda, tell me, in my notes here, we say something about light lace in fiber optic tech.

What precisely is that?

So light lace is the name of our technology.

It’s a fiber optic sensor.

But unlike other fiber optic sensors, it’s soft and stretchable.

And that’s why we’re applying it to humans.

Because if you’re wearing a wearable device, you don’t want to have bulky or uncomfortable fibers or electronics on you.

So that’s our technology, it’s a soft and stretchable fiber optic sensor.

And it measures things like motion.

It measures your missile fatigue.

It can measure your chest expansion, respiration, heart rate.

So basically it is a way to track your vitals as well as your biomechanics while being comfortable.

Whoa, because this reminds me, I haven’t heard this reference lately, but Chuck and Gary might remember.

There’s an expression that says, it was all over you like a cheap suit.

All right, and a cheap suit doesn’t fit you well, but it needs to in order to look good on you.

But if it doesn’t fit you well, you move one arm and other pieces of the cloth move and it climbs up your neck.

And so what you’re saying, not to put words in your mouth, is that previous suits were wearing you.

Whereas now you get to wear the suit.

Is that a fair characterization of this?

I think we can say that.

And it’s like one of our selling points is that you wear compression garments or like tight fitting clothing to exercise anyways.

So it’s just gonna feel the same way, but now we’re making them smarter.

Right, so everything in Lululemon is snug fitting on everybody’s body who buys clothing there.

So now it’s gonna know all about you, is what you’re saying.

How is the data retrieved?

Like, I mean, what is this?

How is this speaking to whatever it’s speaking to so that we get all this information?

Yeah, it’s pretty similar to other wearable devices.

We have a phone app, like an iOS app you can download and it speaks to the device via Bluetooth.

So you can just look at your measurements real time or stop your exercise and just log in and see how you did.

So when you put the fibers through fabric, is there an optimal pattern for those fibers to be woven into?

Or is it basically a simple grid system?

There is an optimal pattern because our fibers are made so that only parts of the fibers are sensing and we want the sensing mechanism to be where we want to sense.

So if you want to sense your tricep, the fiber has to be, the sensing part of the fiber has to be woven into or integrated into the garment in that specific area.

If you’re sensing your chest extension.

I want to monitor my belly fat.

We’ve done something where we place our sensors around the abdominal area to measure how you’re stiffening your muscles when you’re doing certain exercises.

So what we could do is you could wear this every day and it will eventually tell you if your belly got larger in size or it got smaller in size or it’s stiffer when you touch it.

You know, we could also measure the response back when you touch it, so if it’s soft, we can measure that.

If it’s no longer soft, we can measure that.

Mm-hmm, mm-hmm.

Yeah.

Okay.

Are you able to weave this into fabrics that are very slight, high performance, say like for a dancer, a performer, an athlete, or do the fibers, and you said they’re the sort of diameter of a human hair, are they necessary to be in thicker, more heavyweight fabric?

Or can you really go into lightweight?

We can’t go into lightweight, but just a clarification, the fibers we currently make are not as thin as human hair.

It is possible.

Currently, they’re around 800 microns in diameter.

And we prefer actually thinner fabric for them to go into because like, especially if we’re going to wear it outside, then somewhere you don’t want to wear something thick, but we could do either.

So right now we’re talking about diagnostic, and if I’m getting ahead, Gary, then-

So instead of listening to the body, which is what you’re doing, is there an opportunity to tell the body what to do?

So we can tell the human that they need to do something about it, but we’re not trainers ourselves, so we’re going to have to rely on a trainer to look at the data and say, oh, okay, when you do this training five times in a row, 10 minutes intervals, this is how your body reacts, so maybe you should change this, and then they can take more data and see how that change affected that response from the body.

Oh, and you get real-time data, multidimensional data there.

That’s excellent.

I think Chuck wants to know about actuation and how you can modulate the force, but sensors are hard enough, so we’re going to do that first.

There you go.

You’re in the sci-fi mode there, Chuck.

There are people working on it.

It’s very hard.

My lab is working on it, but for a company, it’s very difficult.

But if you could, then it could be like the force.

You can go on your app and just change what the body, the human body is doing in response to your wants and needs.

Yeah, that’s one of the things the bioastronautics application is for.

It’s the sensors for measuring the environment, and they’re also built-in actuators into the mechanical counter pressure suit to help them perform better.

Right, right, right.

Wow, that is crazy.

So, back to what Ilayda just mentioned, when you’re talking about we have this tranche of data, and then we offer this reservoir of data, is it possible at one point, you’ll be putting personal trainers out of work, because that data will be uploaded to an AI cloud, and it will make those determinations for you.

Oh, yeah, I like that.

I like that.

I mean, I think this can go, we can say that for many other jobs, like a lot of technologies have the potential to take over or complement it.

We want to be on the side of complementing the personal trainer or physical therapist or a coach, but we’re not trying to take anyone’s job.

I see what you’re doing there.

I see what you’re doing there, Ilayda, because you’re like, look, these personal trainers, they’re pretty large people and they’re very, very fit.

We don’t want to piss them off.

We don’t want to piss these guys off.

So, the thing here is, Major League Sport has already taken note of light lace.

And you’ve, congratulations, by the way, were awarded an NFL award for $50,000, first and future award for your product.

What was it that they saw in light lace that made them so enamored?

And you’ve kind of shined a light pun intended on that for us.

We get a lot of light puns, for sure.

You know, there are a lot of selling points of our product.

I think one being we can measure different things at the same time.

Instead of relying on like three devices, you can just use one shirt that has light lace and it can measure both your respiration, your muscle fatigue and your motion.

So, if you want to do this currently, you would need three different devices.

So, maybe like a strap that does respiration, you would need sensors that go in your muscles to do that and then you’d need camera based systems for motion tracking.

We combine all of it.

I think there is a lot of value in that.

And because we’re an optical based technology, we have higher sampling rates.

So, if there is inconsistency between each movement, we’re more likely to measure that compared to any other sensor.

So, I think that was another reason why they were interested.

And lastly, I also add this.

This is a shirt like we have some shorts behind me, but it doesn’t have to be a shirt.

And for NFL, for example, leggings, lower body, those are also very important.

And it’s very, very simple for us to just integrate the sensor into leggings or lower body garments and measure that as well.

So, there’s a potential to measure full body, gloves, shirts, leggings, socks, shoes.

It doesn’t, depending on what the person wants to know, we don’t really have a lot of limitations when it comes to the form factor.

How well do you see, Rob, your light lace will perform in an NFL game when it comes to contact?

Because you have pressure sensors here.

Well, yeah, and it’s contact and weather conditions as well because you’re playing in every single conceivable weather condition in the NFL.

Plus you’re sweating.

And you’re sweating.

On top of your fiber optics, yeah.

And you’re having a car collision on every single play.

Right, as they describe it.

Yeah, they describe the impact of every tackle as though you’re in a car accident every single time.

There are rapid decelerations, for sure.

We can capture those.

As Ilayda mentioned, we can sample really, really fast.

Think of this as a wearable, ultra high-speed motion capture system that can also measure pressure interactions.

It has to be worn close to the body.

So, the temperature variations aren’t that big a deal.

Because your body is going to maintain that.

It’s on body temperature, right?

It’s not electrical, so you can get it wet.

There’s not really that many environmental sensitivities for using sports, and they can handle high pressures there.

They’re soft, but they’re very tough.

And toughness means you can absorb a lot of energy before you break.

So, you know what this sounds like?

If you have a full body lace, everything, right?

Neck, everything.

And then you go up with your iOS app, that’s a tricorder, isn’t it?

That sounds great.

Isn’t that a tricorder?

The tricorder measures everything that’s going on in the body without cutting you open.

That’s kind of what you’ve done there.

We don’t have to just use our sensor.

We can fuse lots of different sensor information.

Your Apple Watch uses photo plus osmography, which shines light in, measures the reflected light out.

And from that, they can, you can get a lot of things.

You can get systolic pressure.

You can get respiration rate.

But what those systems are missing is…

Also, you get your ox numbers too, right?

I mean…

Yeah, pulsed oximetry.

The pulsed oximetry, right.

The amount of signal processing that goes on to get all that information from that little bit of blood flow is incredible.

What is not there is the amount of air inhaled, which is extremely important.

You can calibrate that by seeing the expansion and contraction of the chest cavity, can’t you?

We can.

So yeah, that’s part of it.

So this is better than a tricorder.

Without a doubt.

You have exceeded 25th century technology right here.

Look at you.

If we take a step back, we take a step back, you said this is more accurate than the high-speed cameras.

Now, we’ve done shows in the past discussing high-speed cameras, and it was exclusively on baseball and in particular, pitchers.

Are you now being co-opted to bring this product to baseball, be it college, be it minor league or major league, and are you working with the pitchers?

If you can outgun a high-speed camera, you’ve got to be in there, surely.

Baseball is going to be our beachhead market.

We think there’s a lot of value there because they move really fast, especially pitchers.

And even with human eye or motion capture, camera-based motion capture systems, it’s hard to capture that because everything happens in less than two seconds.

So if you take a camera-based system and you get maybe like four data points throughout the pitch, we can give you, let’s say, 400 data points.

So you don’t have to interpolate between the pitch.

You can just look at the pitch and you’ll be able to see, oh, pitch one versus pitch ten.

At this millisecond you behave differently and maybe that’s why you’re not pitching as fast or maybe that’s why you’re going to get injured.

More importantly, well, not more importantly, just as important is form for everything.

Yes.

Form is all of sports.

It’s all about having that form become a non-conscious second nature.

Muscle memory.

Muscle memory, yeah.

Thank you.

Muscle memory.

You guys are actually able to teach muscle memory.

So there’s a study in 2015 by the first author, I think is Wilk, and what they showed is that after they sampled 300 baseball players, the ones that had the greater shoulder mobility were four times less likely to need Tommy John surgery.

So if you can help them increase their external, internal rotation degree of freedom, then they can increase their lifetime with some reasonable probability.

So teaching form is really important, and these things, a pitch takes two seconds, but the initial accelerations take milliseconds, and it’s obscured.

So if you are trying to capture it with external cameras, those cameras need, you really, and these are, we’re talking about 10,000 degrees per second shoulder rotations.

If you’re capturing at a thousand samples per second, you’re under sampling, and you really need to be sampling at maybe 10 times the rate for a non-periodic motion.

There’s something called a Nyquist criterion, which says twice, but that’s for a periodic motion.

For something aperiodic like this, it needs to be even more than that.

Just to clarify, so if you have something that repeats, and you want to measure it, you can say, how many data points on this repeating feature do I need to characterize exactly what’s going on?

And that’s your Nyquist frequency you’re describing.

But if it doesn’t repeat, you can’t all venture off.

You can’t compare what doesn’t repeat.

And that’s an important point.

We think that pitching inconsistency is a huge variable that cannot be measured right now to what matters.

This is a hypothesis and we need to validate it.

But we think by measuring faster, you will reveal that the accelerations are faster than are being measured right now.

And then if you don’t know it and you can’t repeat it, you can’t get these consistencies.

And further, the torques generated in this, we think are being radically under reported.

And that can result in injuries.

And you wonder why can’t people tell when these injuries are going to happen.

You should know the soft tissue loading conditions that are viable, but pitchers enter themselves all the time.

Coaches are going to love this.

You know, this is even outside of sports.

Yeah.

I’m just thinking about these applications for rehabilitation, not of athletes, but I’m talking people who have catastrophic injuries.

Where they have to learn again how to walk.

Everybody thinks that we walk.

You learned how to walk.

You actually learned how to do all the things that you do, that you take for granted.

Your brain is running, you know, a process that lets you do these things.

And if you damage your brain, then it doesn’t work anymore.

You have to relearn it.

This kind of technology could greatly reduce the amount of time it takes to do that.

Yeah, definitely.

There’s rehab applications, physical therapy applications.

It’s just a way to get feedback.

You know, if your brain can’t get that from your sensors, you can use external sensors like light lace and see that feedback on an app and then try to behave that way and see how change of behavior, change of the way you’re moving, change of your motion is affecting that light lace reading.

One of the themes here is that we have sensors, we can feel ourselves, but we have imperfect memories and biased memories.

And so, being able to digitally record…

I constantly think I’m 6’2.

But anyway, we have these…

If we can digitally record this information, then we can go back not just with ourselves, but with other sports scientists and physiologists and help you with less bias, interpret what’s happening.

And physical therapy is a great application where I’m horrible at doing physical therapy.

I’ve never successfully completed a physical therapy regimen.

And I think if I was reporting my information to the doctor, they would be able to call me out on not actually doing it and help me improve faster.

And I just want to highlight for something, because you just said it, I want to make it clear.

When you refer to the acceleration in a pitcher, just what’s going on, the ball is going from zero miles an hour to a hundred miles an hour in a fraction of a second.

All right?

And the pitcher gets ready, and then they cock their arm, and as their hand goes forward, they’re accelerating the ball from zero to a hundred miles an hour.

There’s nothing we do in life that accelerates that fast.

In a car, your head would snap off and roll backwards.

But there’s not only that, there’s also some rotation of the wrist.

There’s a lot of joint action going on there, and you capture all of that.

I’m very impressed by what this is and what it can do for us.

Thanks for saying that.

It’s really Ilayda who identified that beachhead market, and I’ve been fascinated with it.

And yes, in two seconds, a hundred miles an hour, but how fast does it get in the first 80% of that velocity?

What time duration does it achieve at?

And we think it’s backed up by some other data that it happens very soon in the pitch.

And then this is also why people with longer arms are viewed as better pitchers because they can apply that acceleration for longer periods of time, or they can do the same speed by accelerating a slower rate over a longer period of time, which can preserve their joints for longer.

But we have examples of pitchers who defy that.

And then how do they defy that?

Like Pedro Martinez was not a tall pitcher for the Red Sox.

I think he might have been sub six feet even.

Tim Linson was 5’10, I believe.

And if you look at his biomechanics, he extends his, most pitchers will pitch over something like 85% of their wingspan.

They’ll move that.

But he did it like 125% or something like that.

There’s all kinds of stuff to go in for biomechanics and injury reduction.

And it’s great that systems like, what are they, Trackman and…

Hawkeye, Trackman.

All of these motion capture systems that are out there now and some are even getting higher in sampling rates are providing all this information.

But being able to have that unobscured at rates, I mean, there’s really nothing limiting it.

We could potentially do millions of samples per second.

And they all use cameras.

They all use cameras.

You’re at the actual source.

Yeah, exactly.

There is not a sport that will embrace detailed data like this, like baseball.

The coaches and the GMs are going to love you.

They’ll eat it up.

We got to take a quick break.

When we come back, we’re going to find out what the future of all of this can be in the third segment of StarTalk Sports Edition when we return.

We’re back, StarTalk Sports Edition.

We’ll talk about the future of monitoring what your body is doing on the inside, but you’re doing it from the outside.

This magic material that our two guests have pioneered, and I think I perfected your name here, so you have Ilayda?

Ilayda.

Ilayda.

Yeah, oh good, Ilayda Samilgil.

And Rob Shepherd, you guys are mechanical engineers, but you’re into robotics, you’re into robotic monitoring, the human condition, and sports has a huge benefit from this.

Particularly, which we spent time in the last segment, the rate at which you can gather information on something that is moving fast and is non-repetitive is without precedent here.

So, what are your best applications today?

And what do you see coming down the pike tomorrow?

Yeah, I think as we briefly discussed earlier, the best application, Brighton, would be baseball and pitchers specifically because of the high speed motion they go through and how often they go through that motion, which usually results in injuries.

It’s very common to have a Tommy John surgery there.

But whether we see this in the future, like I said, we want to make full body garments, not just shirts or not just straps, but also leggings, shoes, socks, really anything you want to measure or any area you want to measure.

And we want to apply not only to sports, but also provide medical benefits like clinical application, physical therapy.

We’ve looked into robotics, of course, and even car seats.

Yeah, we had interest in measuring comfort levels, driver attentiveness.

Did you forget your child?

It happens a lot in the car.

Hey, hey, hey, it only happened twice.

And the third time it was on purpose.

No judgment.

Yeah, look, Ilayda, when you discuss those applications outside of the sporting realm, I’m sitting here wondering, and I’m not a computer gamer, but this sounds like it is absolutely set to walk itself into VR and AR.

Now, has that universe come to you?

I’m surprised I forgot to mention that, but the why is there are two different routes we can go for when it comes to AR VR training.

We can do training.

So instead of just seeing a reconstruction of your hands or your body in the VR environment, you’d also receive feedback on how much force you’re applying.

So if you’re learning the medical training, you need to know how much exactly to push when you’re doing a surgery.

And you could get that measurement with our sensors if you were wearing them.

And then gaming, the same thing.

Instead of just seeing your body in the VR environment, you can get feedback on how much you’re pushing, how much you’re pulling something, what that results in, like in the gaming environment.

So now, this is information flowing in one direction.

Do you have the ability to send information back?

So you know, if I’m talking for corrective purposes.

So if I’m throwing and your fabric is saying, hey man, what you’re doing right now is you’re stressing right here on this part of your rotator, boom.

And if you continue to do that, we know when those are going to happen.

But at the point where it’s happening, is there a signal that can be sent that lets me know, oh, I just did that.

And then if I don’t feel that, oh, that’s, I did it right.

That type of deal.

Yeah, there are a couple of different ways we can do this.

One of them being we can send the information back via different light patterns.

So our sensors glow, we can make them not glow or we can make them glow in different patterns.

So maybe having them glow three times or having them glow red could mean you are about to get injured.

You should stop.

We could have them glow green to mean something else.

And the other way to go with this is have the app tell you.

So if you have your phone or iPad or the device in front of you, it could also just tell you there.

This is great feedback, Chuck.

We’ll build that in there.

The other thing we can do is put a little vibrotactile thing in the pod, like you said.

This has an electrical portion too, which is located, a small pod located depending on the environment.

You’re going to make garments glow and change color.

You’re going to sweep up the teen market.

You’re going to get the 20-somethings.

You’re going to get the 30-somethings and the 40-somethings.

You still think they’re 20-somethings.

Not to mention all the ravers that will go into a club and get rid of their light sticks so that they can just use their shoulders.

That’s what I’m saying.

So, look, I’ve got to ask you now, and what you’re doing is incredible, but now, is there a dialogue between your sort of…

Gary, I find what you’re doing entirely credible.

It’s not incredible.

It’s entirely credible.

Thank you.

That’s great.

I stand corrected or sit corrected.

But the work that biomedical engineers are doing, trying to map and code signaling the neural signals, how long before you guys talk to them and you come up with something together?

I mean, from my…

So my lab, this is where we both work on wearables and robotics, which is prosthetics.

And so measuring signals from the outside world and transmitting it to our body through electrical impulses is something we have worked on in the past and are continuing to work on, but it’s of such a long slog and…

Well, electrical impulses that you feel on your skin or that you somehow impart into the brain?

That you would impart into sensors in your skin, which then would send that information to the brain.

Right.

Yeah.

It’s almost the same thing, right?

Because your brain is sensing the rest of your body.

Yeah.

And we worked on the hardware part, which is making a prosthetic hand that actually is pretty good, and it’s lightweight, high-force, it’s quick, feedback controlled and everything.

And it’s quite simple, at least for us, to add the capability for that hardware to output electrical impulses that could be then used.

So something I saw that you’ve been working on is embodied energy systems.

And I’ve seen a little tiny little video of your soft aquatic robot, which I would call a fish.

But it’s been called a soft aquatic robot.

So talk to me about embodied energy systems, because I think that’s going to talk Chuck out.

So if you look at the best example of the general purpose robot today, I think is Spot from Boston Dynamics, that yellow quadruped that walks around.

So we can do that for about, at least the last time I heard is 90 minutes.

Is that the one that opened the door, two of them opened the door and the other one walks through?

Yeah, there’s appendages now that have hands and so it can, yeah, it’s really useful, fed robots for a long time, but they’re very specific.

This one you can tune to do lots of different things, but it can only do it for 90 minutes.

Where if you look at something, I like to use a walrus as an example.

A walrus has a ton of fat on it, but that fat is multifunctional that you can, a walrus can operate without recharging for weeks at a time and it can just sit still for theoretically months at a time, but it can do all kinds of agile things.

It could outrun you, definitely me.

They’re fast on land.

They can swim.

They can do all kinds of things.

They can do it because they’re using their energy in a multifunctional way.

It’s not just a battery pack that isn’t adding anything else to the system.

Wait, Rob, if a walrus is chasing me on the land, it’s not catching me.

I would never live that one down.

This one is not going to catch.

They’re pretty fast.

They’re pretty fast.

They’re like, no, think of a hippo in the water.

You would think big fat ass hippo will never catch me.

But you put a hippo in the water, those suckers are moving like a little motorboat.

I know, I’m just saying, I would never live it down the next day.

Your willpower would chase me down on land, okay?

So we’ve decided to make liquid that could be used as hydraulic fluids for moving robots around also as the battery.

So the liquid has electrical potential that we use to power the robot.

So we’ve called it robot blood.

And now we recently published something called the robot heart, which is a stretchable pump that behaves, it’s electrical.

It’s not the same way as our body works.

There’s no muscle.

Our heart is electrical too, though, Rob.

That’s a good point.

Thank you.

That’s great.

I should start using that.

Stretchable soft pump that it also pumps the same blood that powers the heart and then gets everything moving.

So the energy is doing a lot more than just powering the system.

You’re creeping me out.

You’re creeping me out because now you got robots that have a heart and they got blood and they can feel and they don’t need to be recharged.

Wait, wait.

Chuck, Chuck.

If it bleeds, we can kill it.

If it bleeds, we can kill it.

That’s a good point.

Yeah.

You’ve built a synthetic vascular system into your robots as well.

So, if this fluid is energy dense, it’s not just as a hydraulic, is it creating?

How is it creating energy?

It’s got to be a conductor and hydraulic.

Is it kinetic or is it chemical or is it both?

It’s electrochemical.

So, the hydraulic liquid has electrical potential in it and then we pass it by electrodes which then turn that electrical potential into electricity.

That’s very clever.

Thanks.

And Chuck, any liquid will have that property, not the electrical properties, but it’ll have the pressure properties.

So, he’s just being clever and double-dipping on the utility.

That’s what I’m saying.

It’s like our circulatory system, it does many different things.

It does many different things.

So, it’s making use of one thing in several different ways.

Which is-

That’s what nature does.

Yeah, so that’s very, very clever.

And by the way, stop it!

I’ll tell the students, stop.

Is there anything you could tell us about where you’d like to look at going in the future?

What your aim is, where your aim is?

Well, in my lab, we want general purpose robots.

And for that, they need to be adaptive on planet environments.

Space exploration is a great place for that.

Operate for long periods of time without needing to be recharged.

Also ocean exploration.

So we’re focused on ocean exploration robots and space exploration robots.

But to be adaptive, you need feedback control.

It’s not just vision.

Like, you know, there’s, I don’t think there’s an example of an animal that only uses vision.

I mean, touch is an important part of adaptivity.

So that’s why we spent a lot of time making our robots adapt with these stretchable fiber optic touch sensors.

That makes sense.

Because once again, talking about nature, you know, like so many animals in nature use sensors that we would never even begin to understand.

Well, we would never understand them for our use because we couldn’t do it.

Even like something as simple as a snake sticking out its tongue, you know.

Or a dog, you know, sniffing another dog’s butt, you know.

Really?

You had to go there?

No, come on.

Let’s be honest.

They’re not just perverts.

They’re just a lot of information.

Chuck, Chuck.

The question for the ages is if dogs can smell things like miles away, why do they have to get within a quarter inch of a butt to smell it?

They are perverts.

Doesn’t that mean?

It’s like, hey, you know what, Jim, I knew it was you from a block away, but what the hell?

I’m going to double check at a quarter of an inch.

I figured I’d get up in there anyway.

I knew it.

So, guys, we got to land this plane here.

Before we do, I need a practical answer.

Just how washable is it?

Are we talking one wash?

Two washes?

Can I get a year out of this?

Can I iron it?

We need more tests to be able to say that, but we’ve done tons of tests and they survive.

The electronics part is fully removable, so you can just slide it out and then put it in the washer and dryer.

But to be able to say if it will last a year, we need to test it for a year, which we haven’t yet.

As long as you don’t mind me asking.

And one last point.

I just want to verify, and I think it’s true.

Rob, you were describing the limits of the pitching motion that would put a pitcher at risk of requiring Tommy John surgery.

That presumably you don’t know any of this in advance.

You have to teach the system what will and will not be the consequences of what someone does in the system to then have a baseline of data so that you can advise the next generation of people on what the causes and effects are of their problems.

Isn’t that correct?

You need actual people to try this, get the Tommy John surgery and say, oh, here’s why you needed the Tommy John surgery.

That’s after the fact so that everybody that comes later, they can know it before the fact.

Yeah, we need to provide a benefit that is not just on injury prevention to get people wearing these so that we can develop the probabilistic models that will allow us to predict when these injuries will happen.

But there’s also the possibility that we can analytically predict this based off of, again, we think that accelerations are being radically under predicted based off of motion capture data and if we can say that the acceleration is actually more than 500 degrees per second squared.

It’s 700 in the first couple of milliseconds and greatly that will change.

Just to be clear, that’s an angular acceleration.

Correct.

Yes.

When you say degrees, right.

And that could end up with torques that are 40% more than what people think and like no wonder you’re tearing your shoulder, musculature, ligament, and not just the elbow, although the UCL is what fails mostly.

But then that’s because that acceleration is being transferred into that elbow snap.

So we think that yes, we want a probabilistic model, but maybe along the way we can capture information at high enough frame rates that we can actually understand just from a fundamental level.

And of course, you’re breaking your ligaments because you’re applying torques that are beyond what they can handle.

Wow.

Take Nick.

Hi, guys.

It’s been a delight to have you as guests on StarTalk Sports Edition.

We’ll be monitoring your space going forward.

And if you make new developments, be sure to come back on and talk about them.

Please do.

Right here on StarTalk Sports Edition.

All right.

Excellent.

Chuck, Gary, always good to have you there, man.

Pleasure.

Pleasure.

As my co-host.

This has been StarTalk Sports Edition.

All about the future of stuff, figuring out what you’re doing inside your own skin.

For better or for worse.

I’m Neil deGrasse Tyson, your personal astrophysicist.