I would say yes even though it is not as physically demanding as other sports. Ping Pong definitely requires a great deal of skill; professional players are crazy!
There is a difference between a sport and a game to me.
A sport must require a certain amount of physical activity (strength, power, endurance, etc.)
A game can be extremely hard, but does not have to require much physical activity.
To me, Ping Pong is a game. It's extremely difficult to play at a high level, but I don't think that qualifies it as a sport. If being difficult was all that was needed to classify something as a sport, than playing video games and cooking could both be sports.
For the record, I don't think golf is a sport either. 😛