frm.mwz
Well-Known Member
From my own experiments on a 2CPU/2GB instance at Vultr, I can send 10,000 emails in a little under six minutes.
To get this optimal setting, I changed "Subscriber batches in parallel" to one under Settings/Cron. I used the default of 500 in "Subscribers at once" as a baseline. Then, I ran a dummy campaign from the command line using this command "php apps/console/console.php send-campaigns --campaigns_type=regular --verbose=1" to gauge how many emails I could deliver from MW to my MTA within a minute. Once I found my sweet spot of 600 to where the cronjob completed in under a minute, I change the "Subscribers in parallel" back to its defaults. In order to keep the PHP processes from stumbling over each other fighting for CPU, I reduced "Subscribers in parallel" to four which left resources available for other systems services.
I'm guessing when running multiple campaigns, depending on your CPU, you'll need to adjust "Subscriber batches in parallel". For example, if I were running four different campaigns consistently, I'd reduce "Subscriber batches in parallel" to one to allow four PHP processes instead it possible running sixteen (twisted1919, correct me if this is wrong).
Apache Compile Settings:
Code:Server version: Apache/2.4.18 (Unix) Server built: Feb 9 2016 12:07:40 Server's Module Magic Number: 20120211:52 Server loaded: APR 1.5.2, APR-UTIL 1.5.4 Compiled using: APR 1.5.2, APR-UTIL 1.5.4 Architecture: 64-bit Server MPM: event threaded: yes (fixed thread count) forked: yes (variable process count) Server compiled with.... -D APR_HAS_SENDFILE -D APR_HAS_MMAP -D APR_HAVE_IPV6 (IPv4-mapped addresses enabled) -D APR_USE_SYSVSEM_SERIALIZE -D APR_USE_PTHREAD_SERIALIZE -D SINGLE_LISTEN_UNSERIALIZED_ACCEPT -D APR_HAS_OTHER_CHILD -D AP_HAVE_RELIABLE_PIPED_LOGS -D DYNAMIC_MODULE_LIMIT=256 -D HTTPD_ROOT="/usr" -D SUEXEC_BIN="/usr/bin/suexec" -D DEFAULT_PIDLOG="/var/run/httpd.pid" -D DEFAULT_SCOREBOARD="logs/apache_runtime_status" -D DEFAULT_ERRORLOG="logs/error_log" -D AP_TYPES_CONFIG_FILE="/etc/apache2/mime.types" -D SERVER_CONFIG_FILE="/etc/apache2/httpd.conf"
PHP Compile Settings:
Code:PHP Version => 5.6.17-pl0-gentoo System => Linux ajuntapall 4.1.15-gentoo-r1 #2 SMP Thu Feb 4 10:34:04 EST 2016 x86_64 Build Date => Feb 4 2016 06:40:11 Configure Command => './configure' '--prefix=/usr' '--build=x86_64-pc-linux-gnu' '--host=x86_64-pc-linux-gnu' '--mandir=/usr/share/man' '- -infodir=/usr/share/info' '--datadir=/usr/share' '--sysconfdir=/etc' '--localstatedir=/var/lib' '--prefix=/usr/lib64/php5.6' '--mandir=/usr/ lib64/php5.6/man' '--infodir=/usr/lib64/php5.6/info' '--libdir=/usr/lib64/php5.6/lib' '--with-libdir=lib64' '--without-pear' '--enable-maint ainer-zts' '--disable-bcmath' '--with-bz2=/usr' '--disable-calendar' '--enable-ctype' '--with-curl=/usr' '--enable-dom' '--without-enchant' '--disable-exif' '--enable-fileinfo' '--enable-filter' '--disable-ftp' '--with-gettext=/usr' '--without-gmp' '--enable-hash' '--without-mhas h' '--with-iconv' '--enable-intl' '--enable-ipv6' '--enable-json' '--without-kerberos' '--enable-libxml' '--with-libxml-dir=/usr' '--enable- mbstring' '--with-mcrypt=/usr' '--without-mssql' '--with-onig=/usr' '--with-openssl=/usr' '--with-openssl-dir=/usr' '--enable-pcntl' '--enab le-phar' '--enable-pdo' '--enable-opcache' '--without-pgsql' '--enable-posix' '--without-pspell' '--without-recode' '--enable-simplexml' '-- disable-shmop' '--without-snmp' '--disable-soap' '--disable-sockets' '--without-sqlite3' '--without-sybase-ct' '--disable-sysvmsg' '--disabl e-sysvsem' '--disable-sysvshm' '--without-fpm-systemd' '--without-tidy' '--enable-tokenizer' '--disable-wddx' '--enable-xml' '--enable-xmlre ader' '--disable-xmlwriter' '--without-xmlrpc' '--without-xsl' '--enable-zip' '--with-zlib=/usr' '--disable-debug' '--without-cdb' '--withou t-db4' '--disable-flatfile' '--without-gdbm' '--disable-inifile' '--without-qdbm' '--with-freetype-dir=/usr' '--with-t1lib=/usr' '--disable- gd-jis-conv' '--with-jpeg-dir=/usr' '--with-png-dir=/usr' '--without-xpm-dir' '--without-vpx-dir' '--with-gd' '--with-imap=/usr' '--with-ima p-ssl=/usr' '--without-interbase' '--with-mysql=mysqlnd' '--with-mysqli=mysqlnd' '--with-mysql-sock=/var/run/mysqld/mysqld.sock' '--without- unixODBC' '--without-iodbc' '--without-oci8' '--without-pdo-dblib' '--with-pdo-mysql=mysqlnd' '--without-pdo-pgsql' '--without-pdo-sqlite' ' --without-pdo-odbc' '--without-pdo-oci' '--without-readline' '--without-libedit' '--without-mm' '--with-pic' '--with-pcre-regex=/usr' '--wit h-pcre-dir=/usr' '--with-config-file-path=/etc/php/cli-php5.6' '--with-config-file-scan-dir=/etc/php/cli-php5.6/ext-active' '--disable-embed ' '--enable-cli' '--disable-cgi' '--disable-fpm' '--without-apxs2' 'build_alias=x86_64-pc-linux-gnu' 'host_alias=x86_64-pc-linux-gnu' 'CFLAG S=-march=native '-O2' '-pipe'' 'LDFLAGS=-Wl,-O1 '-Wl,--as-needed'' 'CPPFLAGS=' 'CXXFLAGS=-march=native '-O2' '-pipe' '-fvisibility-inlines-h idden'' Server API => Command Line Interface Virtual Directory Support => enabled Configuration File (php.ini) Path => /etc/php/cli-php5.6 Loaded Configuration File => /etc/php/cli-php5.6/php.ini Scan this dir for additional .ini files => /etc/php/cli-php5.6/ext-active Additional .ini files parsed => /etc/php/cli-php5.6/ext-active/apcu.ini, /etc/php/cli-php5.6/ext-active/geoip.ini, /etc/php/cli-php5.6/ext-active/ioncube.ini, /etc/php/cli-php5.6/ext-active/opcache.ini, /etc/php/cli-php5.6/ext-active/redis.ini, /etc/php/cli-php5.6/ext-active/suhosin.ini PHP API => 20131106 PHP Extension => 20131226 Zend Extension => 220131226 Zend Extension Build => API220131226,TS PHP Extension Build => API20131226,TS Debug Build => no Thread Safety => enabled Zend Signal Handling => disabled Zend Memory Manager => enabled Zend Multibyte Support => provided by mbstring IPv6 Support => enabled DTrace Support => disabled Registered PHP Streams => https, ftps, compress.zlib, compress.bzip2, php, file, glob, data, http, ftp, phar, zip Registered Stream Socket Transports => tcp, udp, unix, udg, ssl, sslv3, sslv2, tls, tlsv1.0, tlsv1.1, tlsv1.2 Registered Stream Filters => zlib.*, bzip2.*, convert.iconv.*, mcrypt.*, mdecrypt.*, string.rot13, string.toupper, string.tolower, string.strip_tags, convert.*, consumed, dechunk This program makes use of the Zend Scripting Language Engine: Zend Engine v2.6.0, Copyright (c) 1998-2015 Zend Technologies with the ionCube PHP Loader (enabled) + Intrusion Protection from ioncube24.com (unconfigured) v5.0.18, Copyright (c) 2002-2015, by ionCube Ltd. with Zend OPcache v7.0.6-dev, Copyright (c) 1999-2015, by Zend Technologies with Suhosin v0.9.38, Copyright (c) 2007-2015, by SektionEins GmbH
This is great tweaking & testing, @Mxyzptlk FishStix !
There is definitely some room in the pcntl as with some test (and its somewhat erroneous extension) I found on another mailer which uses parallelization as well, about 12k emails went to 3rd party smtp within under 145s. If memory serves me well, it was on 10 parallel send processes. I hope this will ignite more optimization
