That will make sending 2.6x slower, which nobody wants...since it anything slow clogs up any involved systems.
It is MailWizz strength to have those two. If SwiftMailer screws up for whatever reason, what then?
You were wise to have both, keep it that way.