How increase speed for email/minute

Discussion in 'General discussions' started by Ashish Shukla, Mar 30, 2019.

  1. Ashish Shukla

    Ashish Shukla New Member

    Joined:
    Dec 21, 2017
    Messages:
    7
    Likes Received:
    0
    S.E:
    2019-08-22 13:03:57
    L.T:
    Regular
    L.C:
    2
    Hi,
    I am using mailwizz since last 2 years but now I am facing slow delivery speed.
    I have 1 customer account only with 55 Delivery server and want to send 5M emails in just 4-5 hours. I don't want to rotate delivery server as I developed reputation for my IP and domains. Usually it takes more than 12 hours. I know many of you already solved this problem so I am seeking help from your side.
    I have sufficient server configuration with 96GB RAM and 1 Gbps Bandwidth.
    How may I send 5M emails in just 4-5 hours from MW to PMTA.
    Here is current Backend cron setting

    upload_2019-3-30_14-59-31.png
     
  2. majid1f

    majid1f Member

    Joined:
    Feb 23, 2015
    Messages:
    104
    Likes Received:
    15
  3. laurentiu

    laurentiu Member Staff Member

    Joined:
    Mar 13, 2018
    Messages:
    221
    Likes Received:
    24
  4. Ashish Shukla

    Ashish Shukla New Member

    Joined:
    Dec 21, 2017
    Messages:
    7
    Likes Received:
    0
    S.E:
    2019-08-22 13:03:57
    L.T:
    Regular
    L.C:
    2
    Hi @laurentiu
    I implemented all above points but still same issue
     
  5. twisted1919

    twisted1919 Administrator Staff Member

    Joined:
    Dec 27, 2014
    Messages:
    9,980
    Likes Received:
    2,338
  6. Ashish Shukla

    Ashish Shukla New Member

    Joined:
    Dec 21, 2017
    Messages:
    7
    Likes Received:
    0
    S.E:
    2019-08-22 13:03:57
    L.T:
    Regular
    L.C:
    2
    Hi @twisted1919
    Yeah currently I am running manual cron via debug. It's not showing any error or utilization. See below stats

    [2019-04-03 10:21:47] - XXXXX@yahoo.com - 40/50
    [2019-04-03 10:21:47] - Checking if we can send to domain of XXXXXX@yahoo.com...
    [2019-04-03 10:21:47] - Checking if XXXXX@yahoo.com is still confirmed...
    [2019-04-03 10:21:47] - Checking if XXXXX@yahoo.com is blacklisted...
    [2019-04-03 10:21:47] - Checking if XXXXX@yahoo.com is listed in a campaign suppression list...
    [2019-04-03 10:21:47] - Checking if the server is over quota...
    [2019-04-03 10:21:47] - Preparing the entire email...
    [2019-04-03 10:21:47] - Using delivery server: ######### (ID: 3).
    [2019-04-03 10:21:47] - Communication with the delivery server took: 0.8781
    [2019-04-03 10:21:47] - Sending response is: OK
    [2019-04-03 10:21:47] - Sending OK.
    [2019-04-03 10:21:47] - Done for XXXXX@yahoo.com, logging delivery...
    [2019-04-03 10:21:47] - Logging delivery took: 0.00229


    if I am running 50 Subscriber with 20 batches and 60 campaigns at a time. Then it finished within 643 seconds

    [2019-04-03 10:24:00] - This cycle completed in 643.2 seconds and used 2 MB of memory!
    [2019-04-03 10:24:00] - CPU usage in last minute: 13.77, in last 5 minutes: 14.33, in last 15 minutes: 12.96!
     
    Last edited: Apr 3, 2019
  7. Dev1@OptiBiz1

    Dev1@OptiBiz1 Member

    Joined:
    Feb 17, 2016
    Messages:
    37
    Likes Received:
    8
    @Ashish Shukla, I am the dev of https://forum.mailwizz.com/threads/...ge-but-i-want-more-any-ideas.5838/#post-38555

    - Make sure you're not embedding images in the email
    - Make sure you're not attaching files in the email
    - Make sure to use PHPMailer
    - Make sure you are using Redis Cache and Mutex
    - Make sure you enabled campaign queue table
    - Make sure MySQL is configured properly for the type of server you have, example: it should have enough connections, specially innodb variables
    - Make sure your MW server is using at least SSD, much better to use NVME. This affects MySQL performance
    - If your MySQL is hosted in a different server, then latency will be your PROBLEM, specially if it is hosted in a different location in the world.
    - Make sure your MW server network limit is at least 1Gbps, if it is less than 1Gbps, then your sending will be limited.
    - If your MW server network limit is 1Gbps, Your PMTA server network limit should also be 1Gbps. Because if MW server(or PMTA) is lower, then you won't be able to send as much because of network limitation. To check the bandwidth usage while sending email, use iftop command of linux.
    - Make sure your PMTA server is configured to handle many SMTP connections. 1000 is usually ok.

    Now, based on your send-campaigns verbose log, Communication with the delivery server took: 0.8781. That is too much for sending 1 email. What is the email size that you are sending? Is it 100Kb? 200Kb?

    If the email size is just around 20Kb, but delivery server still took around 0.8, then you need to find a way to lower the delivery server communication. This usually happens when MW server and PMTA server is hosted on different part of the world.

    Try checking the latency of your MW server to the PMTA server, is should be close to 1ms as much as possible(for maximum performance). If the latency is around 500ms, then that's a problem. From your MW server, execute mtr -rw [PMTA ip addr]
     
    Last edited: Apr 12, 2019
    twisted1919 likes this.
  8. Ashish Shukla

    Ashish Shukla New Member

    Joined:
    Dec 21, 2017
    Messages:
    7
    Likes Received:
    0
    S.E:
    2019-08-22 13:03:57
    L.T:
    Regular
    L.C:
    2
    Hi @Dev1@OptiBiz1

    Thanks for brief guidance, I solved Delivery speed with Redis. Now facing another issue, once I increased sending limit.

    Process Status = GIVEUP
    Expected response code 220 but got code "421", with message "421 server.domainname.com out of connection slots " Log data: ++ Starting Swift_SmtpTransport << 421 server.domainname.com out of connection slots !! Expected response code 220 but got code "421", with message "421 server.domainname.com out of connection slots " (code: 421)

    SMTP server hosted with different location with another hosting provider. I have updated Max. connection messages = 1 at Delivery server setting.
     
    Last edited: Apr 11, 2019
  9. twisted1919

    twisted1919 Administrator Staff Member

    Joined:
    Dec 27, 2014
    Messages:
    9,980
    Likes Received:
    2,338
    Isn't this pretty clear ?

    You're sending faster than your smtp server can handle.
     
  10. Dev1@OptiBiz1

    Dev1@OptiBiz1 Member

    Joined:
    Feb 17, 2016
    Messages:
    37
    Likes Received:
    8
    3 things to make sure this is resolved.

    - As I have added above:
    - delivery server config Max. connection messages should not be 1, if using PMTA, setting it to 1000+ is fine.
    If it is set to 1, then it will create an SMTP connection to PMTA for every message which is inefficient. PMTA can handle thousands of messages per connection.

    - In Settings > Cron > Send at once, should either be 0 or 1000+, this can also be equal to Max. connection messages.

    If using PHPMailer, Max. connection messages affects how many SMTP connection slots.

    If using SwfitMailer then Max. connection messages and Send at once are related.They will affect how many connection slots was used during sending. Let's say, Max. connection messages is set to 1000, Send at once = 100. Then it It will create SMTP connection for every 100 messages, then once the Max. connection messages is reached, it will create another SMTP connection.
     
    Last edited: Apr 12, 2019
  11. frm.mwz

    frm.mwz Well-Known Member

    Joined:
    Mar 8, 2016
    Messages:
    3,694
    Likes Received:
    675
    S.E:
    2019-05-17 04:02:46
    L.T:
    Regular
    L.C:
    7
    It is actually PCNTL that will determine to a large part how many concurrent connections are being made. Of course, more are needed (or less is sent) if you have "connection messages" (correctly that is really 'messages per connection') set to a lower number (e.g. 1), but in most use cases that is somewhere between fifty and a few thousand.
    Keep in mind that PHPmailer is actually the RFC compliant one (not SwiftMailer), and that it is 2.6 times faster than SwiftMailer (as it does the pipelining better).
     
  12. Dev1@OptiBiz1

    Dev1@OptiBiz1 Member

    Joined:
    Feb 17, 2016
    Messages:
    37
    Likes Received:
    8
    Right, pcntl plays a huge role on the number of smtp conns. Number of campaigns in parallel, subscriber batches in parallel
     
  13. frm.mwz

    frm.mwz Well-Known Member

    Joined:
    Mar 8, 2016
    Messages:
    3,694
    Likes Received:
    675
    S.E:
    2019-05-17 04:02:46
    L.T:
    Regular
    L.C:
    7
    It was about the number of connections (the rest we know).
    Try this: disable PCNTL and see how many concurrent connections you get?
     

Share This Page