why did the CERN folks go ahead with the $6 billion system they have?

Because they want to educate, train and release into Europe qualified and experienced scientists, engineers and technicians, obviously. In this respect it doesn't really matter what the machine and what the science goal is in the end, and clearly they have already been wildly successful at the goal of education, and in this case, publicity.

http://www.xfel.eu/XFELpresse/en/hintergrund/vergleich/index.html

They've already got something going, pretty much everybody does, this is a wide open field for all. No one machine can have a monopoly. For instance, in soft to hard x-rays, we already have the LCLS - Linear Coherent Light Source :

http://www-ssrl.slac.stanford.edu/lcls/

The WIFEL - Wisconsin Free Electron Laser is for VUV to soft x-rays.

I myself am interested in the range of 3 - 10 eV.

A wide variety of frequency combed techniques are already moving into optical spectroscopy so quickly, it's almost impossible to keep up with it, small colleges and universities all over the nation are jumping into it with at least a program.

High energy physics has its place, but if you want to remain economically competitive, light is where the action is. Well, last night's big breakthrough was accomplished with neutrons mostly, but they did follow it up with x-rays.