I don't know of a way to open the VPN dialog box. That might need to be done in C, C++, or C#, I don't think you can with PowerShell.
You might look into ras.h. I am not familiar with C programming, but that header contains the APIs to work with VPN connections in Windows.
I'm not sure what is causing the error, you will need to examine the log files. Maybe an issue during the main mode negotiation? Password mismatch? Possibly some of the encryption choices (AuthenticationTransformConstants, IntegrityCheckMethod, etc) don't match the receiving end.