This popup suggests that Java is not installed at all on the machine. Can you try to install latest version of Java runtime?
For enabling XY mode, try to do it before you get to that page with popup. However, this kind of dialog is not originating from Java, it's from browser itself and WPM will be most likely not able to handle it.