Why not? Golf may not have the long traditions and wide following of team handball, but you have to try new things.
Golf is a world game, and most cities that would host the Olympics already have a course that could function, or build a new course (a great way to use some actually toxic wasteland and clean up the neighborhood. It would not require nearly the reorganization of systems as baseball or basketball did.
Would not be any more professionally oriented than the above or tennis, it could be as pure as skiing. Like skiing amatures can compete on the same course, like skiing they just won't be competitive. Men's and women's competition could easily be established at the same time so no ski jumping problems.
If memory serves golf at least has a history of having been an Olympic sport in the past. Bring it back.