I finally managed to get Selenium tests starting the Chrome Driver on my laptop (server).
The important bit is to use Xvfb
. Don't ask me why but once you accept this fact follow these steps (more detailed than @Anon answer)
In you Jenkins settings add a global property
key : DISPLAYvalue:0:0
On your server start
Xvfb
in the background:Xvfb :0 -ac -screen 0 1024x768x24 &