I'm getting a big server load with requests to Mailwizz's /dswh/{id}

[henkedk]

@twisted1919 I dug a bit deeper on the traces... Whatever PHP function in the DSWH includes these SQL queries is the one taking the time - it is NOT the SQL taking the time, it is PHP itself. But I'm not able to narrow it down more than to find what part of the SQL is being executed while wasting a lot of time.

SELECT `category`, `key`, `value`, `is_serialized` FROM `mw_option` WHERE `category` = :c
SELECT * FROM `mw_language` `t` WHERE `t`.`is_default`=:yp? limit ?
SELECT `data`FROM `mw_session`WHERE expire>:expire AND id=:id
SET names ?
SET time_zone=?
SET sql_mode="
SET names utf?​

 

[frm.mwz]

@twisted1919 @corey34 did you guys ever find a solution here? We're seeing 1-3 full seconds on the DSWH on the PHP side - all DB calls are done within 1-40 MS... So we're eager to figure out what's wrong with the DSWH.

Is the DS which is receiving the WH perhaps slow or overloaded? I cannot imagine mwz being slow here, since it is proven to be able to execute multiple WHs per second resulting in high speed sending if the DS is capable of handling that.

 

[henkedk]

Is the DS which is receiving the WH perhaps slow or overloaded? I cannot imagine mwz being slow here, since it is proven to be able to execute multiple WHs per second resulting in high speed sending if the DS is capable of handling that.

What? Please look at my comment and code - it is a trace of MW PHP, nothing to do with the DS, it is the other way around.

 

[corey34]

I looked back at my old conversations and here is what I did, below. This may not be helpful for your given situation, but this is what helped me.

1. Increase the max connections allowed by mysqlL max_connections = 2000 in /etc/my.cnf
Article: https://mediatemple.net/community/p...-increase-the-mysql-connections-for-my-server
2. Increase the innodb buffer size: follow this awesome article: https://dba.stackexchange.com/questions/27328/how-large-should-be-mysql-innodb-buffer-pool-size. I changed mine to 512M
3. File descriptors: usually you only get arround 1000 file descriptors available.
Run “ulimit -n” to see the actual count you got. If under 65535, change it to that value by running: “ulimit -n 65535”

 

[henkedk]

It is not the MySQL, MySQL part of the trace is ~60ms today, PHP part is 3 full seconds... It is PHP... And ONLY on the dswh... So I assume it is an core problem in MW. That server is under heavy load on a lot of sites, none of them trace back and have bad PHP performance, also MW has good traces on everything but the DSWH...

 

[twisted1919]

o I assume it is an core problem in MW

You assume wrong.

 

[frm.mwz]

What? Please look at my comment and code - it is a trace of MW PHP, nothing to do with the DS, it is the other way around.

There are cases where PHP and SQL are slow since PHP handles SQL relatively slowly. But that does not mean so slow as you described.

It is not the MySQL, MySQL part of the trace is ~60ms today, PHP part is 3 full seconds... It is PHP... And ONLY on the dswh... So I assume it is an core problem in MW. That server is under heavy load on a lot of sites, none of them trace back and have bad PHP performance, also MW has good traces on everything but the DSWH...

Try SendGrid with WH and you will see, it can do over 1k/s

 

[henkedk]

You assume wrong.

I assume right...
apps/frontend/controllers/DswhController.php

/* because posting too fast sometimes can lead to dupes */
usleep(rand(100000, 3000000)); // 0.1 => 3 sec

Keeping a script sleeping randomly from 0.1-3 full seconds bc posting to fast can lead to dupes - that's a core problem!

 

[twisted1919]

That sleep statement exists there for a very good reason as the comment says, it actually helps your database not getting flooded by requests which might end up with duplicates.

 

[Firdaus Azizi]

Any solution yet to this issue? I'm also having high server load since last week, and I'm using Sparkpost also. Not sure what the reason is yet. CPU usage is around 40-50%.

 

[laurentiu]

@Firdaus Azizi - There no issue to be solved here, this is just what happens when sparkpost send to many notification to that endpoint. The only solution is to add more hardware for the problem.

 

[frm.mwz]

Any solution yet to this issue? I'm also having high server load since last week, and I'm using Sparkpost also. Not sure what the reason is yet. CPU usage is around 40-50%.

Adjust your web server (e.g. nginx) to be able to handle many (1-10k+) concurrent requests.