as the base question is about win forms here is the solution : ( I just changed the code by user1032413 to rflect windowsForms settings ) if it's a new key :
Configuration config = configurationManager.OpenExeConfiguration(Application.ExecutablePath);
config.AppSettings.Settings.Add("Key","Value");
config.Save(ConfigurationSaveMode.Modified);
if the key already exists :
Configuration config = ConfigurationManager.OpenExeConfiguration(Application.ExecutablePath);
config.AppSettings.Settings["Key"].Value="Value";
config.Save(ConfigurationSaveMode.Modified);