Born November 6, 1861 James was a Canadian-American. He grew up in Almonte, Ontario and studied physical education in Montreal at McGill University, then moved to the U.S.
After moving to Springfield, Massachusetts, he was a gym teacher at the local YMCA. His athletic director, Luther Gulick gave him 14 days to invent a indoor game that will use "Athletic Distraction."