Powell was born in New York State and his family moved to Illinois in 1851. He attended Illinois College in Jacksonville, and later Oberlin College, where he studied botany and natural sciences. As war approached, he involved himself in a program of self-study, to learn military engineering. In 1861, he joined the 20th Illinois Volunteer Infantry, and was soon in Missouri.
The 20th Illinois was stationed in Cape Girardeau, Missouri, in July, 1861. Powell, then a sargeant, designed and oversaw the construction of the ring of forts which protected the union garrison, evidence of which still exist. At Cape Girardeau, he organized Battery F, 2nd Illinois Light Artillery (composed mostly of Missouri recruits), and was promoted to Captain. While in command of the battery at Shiloh, Powell received a wound which resulted in the amputation of his right arm. He returned to service to fight at Vicksburg and Meridien, and rose to the rank of Major.
Powell achieved his greatest fame for the expedition he lead in 1869, down the Colorado River through the Grand Canyon. He is also considered the father of the U.S. Geological Survey, and served as its director for 23 years. Following his death in 1902, he was buried at Arlington.