After moving around the local Dumfries leagues, the English system and Junior leagues, Annan joined the South of Scotland League in 1977. The club proved very successful in the South of Scotland League, winning every competition that was available to them. In an attempt to get more competitive football, they joined the East of Scotland Football League in season 1987–88, although they maintained their commitment to the South of Scotland League by running a reserve side. They won promotion in their first season in the East of Scotland League, and two years later won the Premier Division. They became one of the league's top sides and qualified for the Scottish Cup's early rounds on various occasions.
Annan applied to join the Scottish Football League in 2000, when two new clubs were admitted, but lost out to Peterhead and Elgin City. Following the demise of local rivals Gretna in 2008, Annan applied along with four other clubs to replace them in the Third Division. They were the successful candidate, being chosen due to the standard of their facilities, ahead of Cove Rangers, Spartans, Preston Athletic and Edinburgh City.