From 1962 to 1963, Elizondo studied dance at the Ballet Arts Company at Carnegie Hall and in 1963 he landed parts in two Off-Broadway shows: Kill the One-Eyed Man and The Great White Hope. His first major success came when he played “God” in guise of a Puerto Rican steam room attendant in the play Steambath. Elizondo won an Obie award for his performance. Many of his roles involve playing a friend or sympathizer to the lead character. As a voice-actor, he played Bane, one of the more aggressively themed characters in Batman: Mystery of the Batwoman. In 1974, Elizondo played against type as ex-mafioso-turned-subway hijacker “Mr. Grey” in The Taking of Pelham One Two Three. Elizondo starred as a Puerto Rican widower on the CBS television series, Popi, which aired during the 1975-1976 television season.⁷ The short-lived series, which ran for eleven episodes, was one of the first American network television series to feature a Latino theme and cast.

In the 1980s, Elizondo befriended Garry Marshall; Marshall was impressed with his talent and it was to become a lifelong friendship, which would bring benefits for both. Their first movie together was Young Doctors in Love. In some of the movies in which Elizondo appeared, he went uncredited and he would not have minded staying that way for the movie Pretty Woman; however, it was Marshall who insisted on crediting him. His role in Pretty Woman only lasted 10 minutes, but it led to his receiving a Golden Globe nomination. Elizondo has participated in over 80 movies and 12 of them have been Marshall's. In 1999, he guest-starred in Runaway Bride as “Fisher” the husband of the male protagonist's ex.

In 2001, he was featured in the short-lived television drama Kate Brasher and portrayed security head Joe in the movie The Princess Diaries, a role he reprised in the 2004 sequel, The Princess Diaries 2: Royal Engagement.

He is probably best known to the television audience as Dr. Phillip Watters on the CBS television series Chicago Hope created by well known television creator David E. Kelley. He has won both an Emmy and ALMA award and was nominated for a Satellite Award and several SAG Awards for playing this role. He is one of only two people to remain on the show for its entire run, the other being Adam Arkin.

On April 30, 2008, it was announced by USA Network that Elizondo would become Adrian Monk's new therapist, Dr. Neven Bell, on the award-winning series Monk. Elizondo replaced actor Stanley Kamel, who died suddenly on April 8, 2008.