When you give to charity, you should pick the charity that will do the most good with your money.
The work you do should be treated the same way. Your skills and time matter, and you shouldn't waste them. You should find the most important thing you can do, and do that.
This doesn't mean everyone has to do the same thing. Obviously everyones' skills are different, but it's also true that any given task can become saturated. No matter how important a task is, if enough people are working on it, your added individual effort might not do as much good as working on some less important, but understaffed project. It's great to be a Doctor, but it's possible for a society to have too many Doctors. On the other hand the skills you have (or could get) must have some field of application which is currently the least well served, relative to it's importance.
This is where you can make the most difference.