Back in the day, brighter colors required rare and expensive dyes. If you saw someone riding into battle wearing bright, clean, bold colors, that tells you you're facing someone with lots of money.

They probably have better weapons and armor than you. They probably eat well and are in good health. They probably dont have to work all day, and have more time to devote to martial training. They're probably going to kill your ###, so you'd better go find someone that looks a little scrubbier.