Silverstein earned four letters as a fullback and quarterback on the football team and was a three-time letterwinner in track. His football career began in 1917 as he scored 42 points, the third-highest total on the team. He also captained the 1918 squad that played just three games due to World War I.
After serving in the war, Silverstein returned to the field for the 1919 season and led the team to an 8-1 season and a Southern Championship. Most notable was his field goal in the final minute against Virginia Tech, giving the Generals a 3-0 win. Following the season, he was named First Team All-South by the Washington Post and Lynchburg News.
Silverstein continued his consistent play during the 1920 season, leading the Generals to a 5-3 season and earning All-South Atlantic Region honors by the Washington Star. That spring while competing in track, he set a W&L record in the discus throw with a toss of 112’11” in a meet against Virginia Tech.
Silverstein was also active around campus during his career. He was President and Chairman of the Athletic Council, was a member of the committee for Fancy Dress, and was inducted into ODK.