“I was doing a movie a couple years ago and one of my bullies was working security on the movie — a guy who literally, one day at school, turned me upside down and shook the money out of my pockets,” the comedian, 57, said onThe Howard Stern Showin September 2020.

Rock said his former bully “was kind of ashamed” upon coming across Rock in the present, but Rock “just gave him a quick look, like, ‘Hey, man, I hope you’re doing well, take care,’ and I kept it moving.”

For more on the Will Smith-Chris Rock incident at the Oscars, listen below to our daily podcast on PEOPLE Every Day.

“I didn’t [say] like, ‘Get him off the set, get him fired!’ His sin has punished him,” Rock told hostHoward Stern.

The comedian went on to say that the man “could’ve been my friend — he could’ve been in the trailer with us watchingThe Godfather. The fact he had to go through that whole day, watch me walk around — I was directing the movie — and watch me be me in all my glory, and to be so close to me but yet so motherf—ing far, I didn’t have to do anything.”

Never miss a story — sign up forPEOPLE’s free daily newsletterto stay up-to-date on the best of what PEOPLE has to offer, from juicy celebrity news to compelling human-interest stories.

Chris Rock.Matt Winkelmeyer/Getty

Chris Rock

He compared the 50-year-old actress' appearance to that ofDemi Moore’s look inG.I. Jane.Jada, who recently opened up about living with alopecia, was visibly unamused, rolling her eyes from her seat. Seconds later, Will walked onstage and approached Rock,smacking him in front of the audienceand millions watching the telecast at home.

Back at his table, Will shouted to a stunned Rock, “Keep my wife’s name out of your f—ing mouth.” Will then remained seated with his wife at his table for the rest of the night. Rock has so fardeclined to press charges.

Will went onto publicly apologize to Rockon social media the following day.

RELATED VIDEO: HowWill SmithSmacking Chris Rock atthe Oscars— and the Aftermath — Unfolded

On Monday, the Academy said in a statement that it “condemns the actions of Mr. Smith at last night’s show” and that the organization has “officially started a formal reviewaround the incident and will explore further action and consequences in accordance with our Bylaws, Standards of Conduct and California law.”

Neilson Barnard/Getty

will smith

Two months before the altercation, Rock expressed that he haslet people “walk all over” himsince his childhood, recounting being “bullied ridiculously” throughout that time in his life on the Jan. 12 episode ofFly on the Wall with Dana Carvey and David Spade.

“Half of the bullying was because I was just a little guy. Then I got bused to school and the bullying was because I was little guy and I was Black. I was getting double bullied,” he said.

Rock went on to say he eventually resorted to taking matters into his own hands: “I went home, I put a brick in a book bag — this is like a legendary story in my neighborhood — I swung that s— and smacked the guy in the face with this brick and stomped on him,Joe Pesci-style, to the point that we thought he might die.”

Later on, with the help of therapy, Rock realized the long-term impact the event had on his life.

“Long story short, from that day on, as my shrink puts it to me, you have been scared to be angry ever since,” he said. “The guy you saw was bending over backwards to be nice because I was so scared of my anger.”

source: people.com