Learning Music (or other extracurricular) Affordably

We have invested a lot to help our kids pick up music when we lived abroad. Books and learning aids were very affordable but since coming back we have discovered that, not only are books double or triple the price, but they are very hard to come by.

Please share your tips on how you have managed to give your children a musical (or any other extracurricular) education without breaking (or robbing) the bank!