Before the 2003 NFL season, Fassel was reunited with a son, Norman "Nemo" Fassel, whom he and his wife Kitty gave up for adoption in 1969. The couple was unmarried when he was born.
In 2004 and 2005, Jim and Kitty Fassel owned a Quiznos restaurant in Denville, New Jersey. Their son Brian was in charge of day to day operations. The store was sold in April 2005 but has since closed.
Jim and Kitty divorced in 2006 after years of counselling.