I always thought Gandalf was pretty gimp in terms of battle power, so he didnt take on 1 on 1 Saurun or Saruman and was struggling eith the balrog (and maybe teh witch king? I dont remember that from the books). His strength was as a stretegist, pulling all the strings and making sure people were at the right place at the right time, doing the right thing.