Anyone here pushing 5M+ daily with mailwizz??

Discussion in 'General discussions' started by dante, Feb 14, 2017.

  1. mpspringer

    mpspringer New Member

    Joined:
    Nov 10, 2016
    Messages:
    5
    Likes Received:
    1
    S.E:
    Expired
    L.T:
    Regular
    L.C:
    1
    Very Impressive Pradeep, Well Done Sir! Would you mind please to kindly enlighten us on particular mailwizz settings and what was modified to increase sending limits that high?

    - what is the biggest bottleneck in mailwizz?
    - are there features/standard configurations that can be turned off to increase send?
    - settings/customers/campaigns >subscribers at once
    - settings/cron > delivery settings (including PCNTL settings)
    - php version
    - where mailwizz files modified?
    - settings/delivery servers > pickup directory? local directory?
    - why/how so many delivery servers?
    - are other extensions used? if so, which ones?
    - is mailwizz dumping flat files into the pickup directory for pmta? if so, at what rate?
     
  2. pradeep sharma

    pradeep sharma Active Member

    Joined:
    Nov 11, 2015
    Messages:
    224
    Likes Received:
    94
    S.E:
    Expired
    L.T:
    Regular
    L.C:
    2
    Thanks for appreciations Guys..
    i would like to tell you that we have almost completely modified Mailwizz (almost 80%) and there is no come back left for me to upgrade the core mailwizz. what i do is follow the recent development in official Mailwizz and then custom code with my developers to match with my code.
    I have also added 50+ Feature which is not available in in official mailwizz.
    Feature that i added :
    1. Dynamic List Include/Exclude(fancy List include /exclude feature)
    2. Multi list include/exclude
    3. CCT tags now work on all Position including Subject Line and From Name
    4. advance List Range Feature like ( like 1000-4000 so only subscriber with stack position 1000-4000 will get included) RANDUM feature is turned off
    5. Campaign Slicing over Days and Hours(example : you can schedule a campaign to 1 Million subscribers for 22 hrs then it will be equally distribute over 22 Hrs with accuracy of 1 min. it means that 1000000/22x60= 757. then 757 email will be released per minute from the system for a particular campaign)
    6. List Import is now very Interactive (after import it tells how many records are updated, added,blacklisted). :List import is highly speedy. admin can deploy as many as server to processe list import as fast as possible. you just need to add a server to my cloud and add my script to this server . it takes less than 2 min for me to add a new server to increase list import.
    7. customer has setting to override update records or not while import
    8. List unsubscribe via email and app both and admin can choose what to apply .
    9. Delivery Server Group facility added
    10. Virtual MTA Facility
    11 Custom Tracking link, unsubscribe link, CDN link to all delivery servers
    12. Tracking link, unsubscribe link, CDN Rotation
    13. Multiple delivery server group on a customer group.
    14. customer can choose single or multiple delivery server group while campaign setup.
    15. Campaign filter dashboard on over 120+ Parameters.
    16. admin campaign filter on 150+ parameters
    17. for All New added subscriber to entire my system will get 2 campaigns from a list clean servers and ips (even return path/CDN/Trackign etc all are changed) before it is confirmed as a safe subscriber for my system, so now No list clean service is needed for me and customer is very happy about it).
    18. Inbox texting/SMTP testing facility for admin
    19. IP reputation Tracker
    20. Admin suppression list (if a subscriber is added to this list the this subscriber will not get any email from any of your customer)
    21 campaign stats refresh via AJAX on campaign filter and list
    22. Ad agency Feature on campaign(customer can now add/create ad agency then add a filter campaign on this field as well)
    23. Custom list field value on EMAIL/campaign Sent
    24. force subscribe from all list as a setting from admin dashboard.
    25. custom X-Mailer Value for each customer if required.
    26. custom Bulk precedence header for each customer.(yes or no)
    27 Import Blacklist check setting and then option to (Allow import, allow but mark blacklist, reject blacklist).
    28. Mailer Size in Kb setting for customer group( if a mailer size goes beyond specified level then system will count is as 2 credit for 1 mail sent, more in details if you specified that 100kb =1 Mail credit the if customer sends 100 emails with 235Kb mailer size then system will count it as 300 Mail credit)
    29. ISP setting on customer group( you can now allow customer to send emails on ISP you allow from backend, like you can now allow GMAIL but discard all other, or allow GMAIL+YAHOO and discard all then all emails and so on.. no need to this at MTA level).
    30. We have a separate CDN server for email images.
    31. Image name and HASH change for every 100 email sent and keep all new images for almost 6 Months then auto delete all images.
    32. Dedicated Open Tracking Sever
    33. dedicated click tracking server (most important)
    34. dedicated list clean server
    35. dedicated unsubscribe /report tracking server.
    36. Dedicated Test Template Server with High Priority Queue.
    37. Test list setting ( customer can now create a list Type testing).
    38. sample email on campaign start and campaign ends.
    39. SMS on campaign start and campaign end
    40. campaign report Export with 80+ columns(both from admin and customer).
    41. customer HTML code which support all TAGS on HTML email template header(footer is already in official mailwizz)
    42. if Credit expire or less then campaign total subscriber then campaign will go for only vaild (paid ) credit subscribers.
    and list goes on many more....
    i too forget..how much i have changed and feature i added.


    IN Pipe Line :
    Predefined Auto List Segment and Filter
    1. On Date added
    2. On last 1/3/5/10//20 campaigns open
    3. On last 1/3/5/10//20 campaigns click
    4. On Segment by Open Month (JAN,Feb)
    5. On Segment by Click month(JAN, Feb)

    Better GRAPHS reporting



    Performance :
    1. Tested with more than 60millon emails in one list (No Dashboard hangup, if you add more than 3M subscriber then dashbaord wont load).
    2. Tested with 6-8 million email per Hr with no issue. i can now scale it any limit (any limit means any limit ) with no problem just need to add few more processing servers. Processing server requires just default CENT OS and my 1 Script File and 1 database entry in my cloud. and it automatically added to my cloud AND starts processing mails)
    3. 30 Million emails import to 1 list in just 10 Hrs.( if you have multiple list then time can be much less).

    Limitations:
    1. Works with PHP version 5.4+ and i am using PHP 7 with NGINX, MYSQL 5.6
    2. Only works and Suits if you have PMTA or any such MTA.

    Infrastructure that i Use for MailWizz

    1. 80 CORE CPU with 265 GB DDR3 RAM and 128GB*12 SSD for application server(NO VPS, we need to modity some system file for better performance)
    2. 8 CORE CPU with 32 GB DDR2 for open Tracking server
    3. 8 CORE CPU with 32 GB DDR2 for click Tracking server
    4. 8 CORE CPU with 32 GB DDR2 for Unsubscribe Tracking server via application
    5. 8 CORE CPU with 32 GB DDR2 for Unsubscribe Tracking server via Mail
    6. 80 CORE CPU with 1 TB RAM for MASTER mysql server 16*128GB SSD
    7. 80 CORE CPU with 256 GB RAM slave-1 mysql server 16*128GB SSD
    8. 80 CORE CPU with 256 GB RAM slave-2 mysql server 16*128GB SSD
    10. ANYCAST DNS server is on cloud (like Route 53 or any other ).
    11 . 10 QTY of Mail Processing servers centos 6.7 with my 2000 Line script (NO virtualisation)
    12. i have 6000+ IPV4 and 6M of IPv6 (/64) as big as google have for IPv6 Block i.e /32 of Ipv6
    13. PMTA Licence or any other way to run PMTA.
    14 I have developed my own PMTA config Generator which generates PMTA config in just 1-2 Min of work.
    15 . Dedicated MTA for Dedicated ISP (for GMAIL/YAHOO/HOTMAIL/OTHERS) we use rerouting feature for this.
    16. 2 Nos of 8core 16GB DDR2 RAM dedicated server for Application zip Backup with 12 TB Space on each server. Automatic backup every day.

    FYI: I have my own DATACenter in India with 100 Servers

    NOTE : Number of IP dont play much role. i have sent 30-40K email per hr per IP for Gmail with no issue. For GMAIL Inboxing you need to maintain all 8 reputation if any one goes down then forget InBoxing, if you still continue to send with low Inbox RATE then forget your domain for almost 1 year+. you will never be able to sent in INBOX for almost 1 year or even more with that domain.
    1. IP
    2. returnpath
    3. DKIM
    4. message ID
    5. CDN/Tracking
    6 CONTENT AND KEYWORDS
    7. SMTP Source HOST
    8. Daily Abuse report SPAM count

    MY Passion is Email IN-BOXING especially for GMAIL and due to God Grace i am doing Good.
    Thanks to @twisted1919 for Providing Such a wonderful application i just further added more bricks and more concepts on current setup to suit my requirement


    My Application Screens are attached
    upload_2017-2-22_16-9-49.png
    upload_2017-2-22_16-10-33.png
    upload_2017-2-22_16-11-39.png
    upload_2017-2-22_16-14-18.png

    upload_2017-2-22_16-15-0.png
    upload_2017-2-22_16-15-59.png
    upload_2017-2-22_16-18-2.png
    upload_2017-2-22_16-26-41.png upload_2017-2-22_16-26-41.png upload_2017-2-22_16-9-49.png upload_2017-2-22_16-10-33.png upload_2017-2-22_16-11-39.png upload_2017-2-22_16-14-18.png upload_2017-2-22_16-15-0.png upload_2017-2-22_16-15-59.png upload_2017-2-22_16-18-2.png upload_2017-2-22_16-9-49.png upload_2017-2-22_16-10-33.png upload_2017-2-22_16-11-39.png upload_2017-2-22_16-14-18.png upload_2017-2-22_16-15-0.png upload_2017-2-22_16-15-59.png upload_2017-2-22_16-18-2.png upload_2017-2-22_16-9-49.png
     
    Last edited: Feb 22, 2017
    Vincent, Michael Wilding, VVT and 7 others like this.
  3. twisted1919

    twisted1919 Administrator Staff Member

    Joined:
    Dec 27, 2014
    Messages:
    10,289
    Likes Received:
    2,391
    @pradeep sharma - simply amazing, congratulations on the hard work you've put into this.
     
    pradeep sharma likes this.
  4. pradeep sharma

    pradeep sharma Active Member

    Joined:
    Nov 11, 2015
    Messages:
    224
    Likes Received:
    94
    S.E:
    Expired
    L.T:
    Regular
    L.C:
    2
    Hi Guys, I received Many messages from many forum members to share my secret about Mailwizz Scaling limiless.
    I have invested my 1.5 Year on mailwizz to understand its bottleneck and how to reconfigure to achieve 6M-10M emails per hr.
    I have modified all core files and added more than 50+ tables to MailWizz DB.
    if i share it with you to scale it up it will not do much help for you to achieve this because i can only share my architecture and concept but NOT CODE.
    even if i share code it would be just a piece of scrap for you without proper its setup and Support.
    i am a passionate Guy and wish to do More into it. I believe that there is no point of reinventing the wheel.
    So What i can offer FREE to every one on this forum is
    1. Share my Experiences and Bottlenecks.
    2. Upto Certain extend my Architecture.
    3. My Knowledge on IP reputation building and google INBOXING Techniques
    4. idea to achieve the same goal.

    I can also offer my Hosted Application Service as a CUSTOMER to Process Mail as per your volume and requirement where my application will only process mails and will connect to your PMAT like setup to final delivery via your IPs and server for some monitory charges.
    after you like above setup then i can further think about migrating entire setup to your servers and can discuss about Monitory things.

    I ll setup ALL DNS & PMTA setup for you free of cost if you take this service.

    Let me know if you are interested.

    Regards

    Pradeep
     
  5. pradeep sharma

    pradeep sharma Active Member

    Joined:
    Nov 11, 2015
    Messages:
    224
    Likes Received:
    94
    S.E:
    Expired
    L.T:
    Regular
    L.C:
    2
    Thanks @twisted1919

    Your Appreciation will definitely add more fuel to my Passion.
    Regards
     
  6. Vpul Shah

    Vpul Shah Active Member Support Staff

    Joined:
    Feb 23, 2015
    Messages:
    759
    Likes Received:
    121
    S.E:
    2019-12-22 13:03:57
    L.T:
    Regular
    L.C:
    1
    Great work Pradeep.

    Really * * * * *
     
  7. frm.mwz

    frm.mwz Well-Known Member

    Joined:
    Mar 8, 2016
    Messages:
    3,708
    Likes Received:
    679
    S.E:
    2019-11-27 02:17:39
    L.T:
    Regular
    L.C:
    7
    great progress!

    very true :)

    do you have any integration with a good crm, e.g. vtiger?
     
  8. dante

    dante Member

    Joined:
    Jan 20, 2017
    Messages:
    166
    Likes Received:
    19
    S.E:
    Expired
    L.T:
    Regular
    L.C:
    1
    [​IMG]
     
  9. frm.mwz

    frm.mwz Well-Known Member

    Joined:
    Mar 8, 2016
    Messages:
    3,708
    Likes Received:
    679
    S.E:
    2019-11-27 02:17:39
    L.T:
    Regular
    L.C:
    7
    to increase this you need to edit the max for this variable in the code
     
    dante likes this.
  10. dante

    dante Member

    Joined:
    Jan 20, 2017
    Messages:
    166
    Likes Received:
    19
    S.E:
    Expired
    L.T:
    Regular
    L.C:
    1
    I see, thats what I thought, so can we customize and change the code without extra fees? I thought that was not allowed lol or do I have to get the extended version for that?

    @pradeep sharma one work bro "SICK SETUP" and here I thought if i could do 10M/day I would be in heaven lol. Added you on skype ty for accepting :)
     
  11. frm.mwz

    frm.mwz Well-Known Member

    Joined:
    Mar 8, 2016
    Messages:
    3,708
    Likes Received:
    679
    S.E:
    2019-11-27 02:17:39
    L.T:
    Regular
    L.C:
    7
    As far as I know, many are doing it, and they post on the forum and ask questions related to it...and the author replies regularly, so, I guess, it must be ok ;)
     
    dante likes this.
  12. theG

    theG New Member

    Joined:
    Dec 1, 2016
    Messages:
    17
    Likes Received:
    0
    S.E:
    Expired
    L.T:
    Regular
    L.C:
    1
  13. Jamie Whittingham

    Jamie Whittingham Active Member

    Joined:
    Jun 27, 2015
    Messages:
    288
    Likes Received:
    45
    S.E:
    Expired
    L.T:
    Regular
    L.C:
    9
    @pradeep sharma ...... your setup is simply amazing.

    I would love to connect with you and share some ideas and tricks and all of the above.

    When you get chance, please connect with me on Skype = dnjame

    thaks
     
  14. Howard

    Howard Member

    Joined:
    Jan 26, 2015
    Messages:
    59
    Likes Received:
    4
    S.E:
    Expired
    L.T:
    Regular
    L.C:
    1
  15. emailprince

    emailprince New Member

    Joined:
    Aug 13, 2017
    Messages:
    5
    Likes Received:
    2
    S.E:
    Expired
    L.T:
    Regular
    L.C:
    1
    sick setup. pm me.. We work something out
     
  16. ccc

    ccc Member

    Joined:
    Apr 15, 2015
    Messages:
    48
    Likes Received:
    10
    S.E:
    Expired
    L.T:
    Regular
    L.C:
    6
    Do you sell the changes you made? If so, please send me a private message.
     
  17. Little Sweet Miss

    Little Sweet Miss New Member

    Joined:
    Aug 26, 2016
    Messages:
    22
    Likes Received:
    0
    S.E:
    Expired
    L.T:
    Regular
    L.C:
    2
    I am interested in this offer. Will send you a Skype invite shortly. Thanks
     

Share This Page