James H. Wilson had had enough. He decided to walk away from the minstrel business for good in the spring of 1902 shortly after having been shot and jailed in Missouri on the terrible, frightening night that his friend, Louis Wright, had been dragged from the jail and lynched. He had a wife and a father in Kentucky and he returned home to sort out his life. The Indianapolis Freeman had published information about the shooting and lynching on March 8th, and just one month later the same newspaper posted a notice from his colleagues in the Richards & Pringle's Famous Georgia Minstrels that not only indicated they missed him but unashamedly pled with him to come back and rejoin the troupe:

Wilson paid them no heed, but once home in Kentucky, he discovered he had new problems to contend with. To keep busy in Nicholasville and to make a living, the best he managed to come up with was to take charge of music at his Baptist Church, sell newspaper subscriptions for the Indianapolis Freeman, become involved in the community's affairs, and try to keep his life as a concert band director and cornet soloist going as best he could from a distance.