For the first time since the arrest, Remy spoke publicly, talking to reporters Monday at the New England Sports Network's headquarters.
Remy, 61, said he was concerned with what the public would think of him and his family and if he could joke around and be himself. He said his friends and family persuaded him to return to work. His family reminded him about his career "and where it came from and where it is."
"I don't intend to be a quitter," he said. "Don't intend to be one now. It's what I do. It's what I know."
Jared Remy was arrested Aug. 13 after he allegedly pushed Jennifer Martel into a mirror. He was released the following day and is accused of fatally stabbing Martel at their apartment.
He has pleaded not guilty.
He has a 4-year-old daughter by Martel.
"It's been very, very difficult," the elder Remy said. "I've run the full gamut. He's still my son."
Remy said he did not want people to forget Martel, saying his deepest sympathies are with her family.
Remy will return to the booth in the next few weeks, but said he will not speak about his son during his first broadcast.