Carotenoids are yellow pigments that turn aspens and ginkgos stunning shades of yellow and gold. Tannins are responsible for the dull browns of some oaks. And since all leaves contain at least some tannin, that's why most of them eventually turn to brown, especially after they fall to the ground.
Of course temperatures come into play as well. Early freezes reduce colorful foilage because the leaves are killed before they can go through this elaborate process. Long, warm fall days and bright sun will create the most colorful fall show.