Process campaigns using Redis Queue.

Discussion in 'Tips and tricks' started by twisted1919, Feb 27, 2015.

  1. twisted1919

    twisted1919 Administrator Staff Member

    Joined:
    Dec 27, 2014
    Messages:
    10,289
    Likes Received:
    2,391
    @Rodrigo Bustos - I don't know how that could happen, that's been fairly tested at large scale without issues. Any change the redis connection was not properly setup, or maybe using same redis database as other apps ?
    Did you tried it with less batches?
     
  2. Rodrigo Bustos

    Rodrigo Bustos New Member

    Joined:
    Jul 27, 2016
    Messages:
    5
    Likes Received:
    0
    S.E:
    2019-11-06 19:12:32
    L.T:
    Regular
    L.C:
    1
    Mailwizz runs in a dedicated server with Redis inside. Just normal OS/Mailwizz updates there are no changes in config, etc.

    Some months ago we found some duplicates in our external smtp but never found the guilty.. our first thought was our MySQL Cluster (Galera, wsrep or anything related to deadlocks) but we couldn't find anything related.

    This time we deactivated Redis just for testing purposes due it reach 100% cpu usage when Mailwizz began to process a campaign and perform batches and it kept like this meanwhile the campaign was running (quite odd). So we tried with different numbers and combinations with same behavior. Finally without Redis it ran perfectly. As last test we leave redis as cache but mutexes and all went fine. So something odd happens when mutex is used to generate batches under unknown delivery setups yet. (number of batches, emails per batch, etc)

    Below an example (1 single campaign running for testing):

    $cat user@domain 2019-04-24.log
    [2019-04-24 13:02:16] - user@domain - 407/500
    [2019-04-24 13:02:16] - Checking if we can send to domain of user@domain...
    [2019-04-24 13:02:16] - Checking if user@domain is still confirmed...
    [2019-04-24 13:02:16] - Checking if user@domain is blacklisted...
    [2019-04-24 13:02:16] - Checking if user@domain is listed in a campaign suppression list...
    [2019-04-24 13:02:17] - Done for user@domain, logging delivery...
    [2019-04-24 13:03:11] - user@domain - 118/500
    [2019-04-24 13:03:11] - Checking if we can send to domain of user@domain...
    [2019-04-24 13:03:11] - Checking if user@domain is still confirmed...
    [2019-04-24 13:03:11] - Checking if user@domain is blacklisted...
    [2019-04-24 13:03:11] - Checking if user@domain is listed in a campaign suppression list...
    [2019-04-24 13:03:13] - Done for user@domain, logging delivery...
    [2019-04-24 13:06:39] - user@domain - 500/500
    [2019-04-24 13:06:39] - Checking if we can send to domain of user@domain...
    [2019-04-24 13:06:39] - Checking if user@domain is still confirmed...
    [2019-04-24 13:06:39] - Checking if user@domain is blacklisted...
    [2019-04-24 13:06:39] - Checking if user@domain is listed in a campaign suppression list...
    [2019-04-24 13:06:39] - Done for user@domain, logging delivery...


    Hope it helps to clarify.
     
  3. twisted1919

    twisted1919 Administrator Staff Member

    Joined:
    Dec 27, 2014
    Messages:
    10,289
    Likes Received:
    2,391
    @Rodrigo Bustos - Thanks for the info, we'll stress this even more to see what could cause it. Meanwhile, you can open a ticket so we can exchange info.
     
  4. Rodrigo Bustos

    Rodrigo Bustos New Member

    Joined:
    Jul 27, 2016
    Messages:
    5
    Likes Received:
    0
    S.E:
    2019-11-06 19:12:32
    L.T:
    Regular
    L.C:
    1

Share This Page