PCNTL and Quota

VVT

VVT

Active Member
Hi @twisted1919 ,

After the duplicate issue was sorted out, I noticed the following issues when PCNTL is enabled -

1. Hourly quota set for delivery servers don't apply. It keeps on sending.
2. When you pause a campaign, it doesn't stop immediately but continues sending several more thousands. I was told this is because the emails are already created in batches and will take time to send out. It's ok if it's just 1 or 2 thousands, but the variation is so high that it sends 7000-8000 emails before it actually pauses. Is there a way can we control this or lessen that variation ?
3. Like point number 2, MW sends ~10% more emails even after the customer sending quota is exhausted.
4. Unrelated to PCNTL, MW allows a customer to downgrade to a plan with less capacity (number of subscribres, lists etc) even when his utilization is more than what that plan offers. Example, plan A offers 1K subscribers, and plan B offers 2K. Customer has 1.5K subscribers and he was a part of plan B. Next month he shouldn't be able to purchase plan A unless his threshold is below 1K subscribers. But this is now possible.

Thanks in advance ! :)
 
@VVT - Good questions ;)
1) Because of the way pcntl works and spawns parallel processes, each process will count quotas separately, thus the issue with the overquota usage for both subscribers and servers. I mean, i don't think it's actually an issue given this is the way things work. We could use PHP's shared memory extension to alleviate this, http://php.net/manual/en/book.shmop.php but this, like pcntl, is a module that's not always installed on the server.
2) This is strongly related to #1
3) Same as #1
4) Well, how a user would feel if once downgraded, 20k subscribers will be removed because they were over the current plan quota?

P.S: I think we actually should do something related to #1.
 
@twisted1919 Yeah, I understand the technical difficulties, but I cannot expect the same level of understanding from my customers especially when there are a plenty of options available for them :D . So it would be really great if we could do something about the quota things. Until then I decided to disable pcntl :).

twisted1919 said:
4) Well, how a user would feel if once downgraded, 20k subscribers will be removed because they were over the current plan quota?

P.S: I think we actually should do something related to #1.
Click to expand...

I actually didn't want to remove the extra subscribers, if they opt for a smaller plan, it should show a message that they're already over quota for that plan. Because saving data is a costly thing. If the customer keeps 1 million data in data base and he always opts for a cheaper plan with 100 Subscribers limit, then it's a loss for the provider. Hope it makes sense :)
 
@twisted1919 So can I expect resolutions for #1 and #4 any time soon ? :)

Also, one more thing - Are you planning to stop supporting send-parallel-campaigns cron and redis ?
 
Great, thanks for the update :)

Take a look at #4 if it makes sense and is simpler to implement - guess that's a matter of some integer comparisons :D
 
@twisted1919
Would be great if there is an update re #1 to #4
What happens if the sending server's capacity is exhausted and PCNTL keeps on sending, will it know which was the last email to really go out (and the rest stays in the queue til new capacity is available, eg next hour)?
 
@frm.mwz - i am working on these still. Delivery server quota is not that problematic, but the customer quota is the one that poses more issues atm.
 
@twisted1919
Ok, but, at the moment, as it is,
what happens
if the sending server's capacity is exhausted
and PCNTL keeps on 'sending' (even if it cannot get it out),
will it know which was the last email to really go out
(and the rest stays in the queue til new capacity is available, eg next hour)
or what would be the result?
 
@twisted1919
Will it retry immediately (while no delivery server is available)
or will it wait until one is available, and then start again (and then count tries)?
 
The retry works like this:
* cron runs -> select current server -> send emails -> emails failed. mw tries picking another server, none available, sending stops.
* cron runs -> select current server -> none available -> stop.
* after X time -> cron runs -> select current server -> send failed email.

If that makes any sense...
 
twisted1919 said:
Because of the way pcntl works and spawns parallel processes, each process will count quotas separately, thus the issue with the overquota usage for both subscribers and servers. I mean, i don't think it's actually an issue given this is the way things work. We could use PHP's shared memory extension to alleviate this, http://php.net/manual/en/book.shmop.php but this, like pcntl, is a module that's not always installed on the server.
Click to expand...
Will enabling "shmop" fix the errors of mwz sending beyond delivery server quotas, customer quotas, and will it make the 'change server at' work? In recent tests it used only two of five DS, and went-on regardless of DS quota finished trying to use that DS (for twice as much, banging on the door thousands of times, which can get accounts closed). I wonder what can be done to make this work. Have you tried using extra control variables in your tests to see where mwz overlooks the limits? How could I help you fix this? I would appreciate this fixed very much :)
 
@twisted1919
So, now with 'shmop' installed, what is the next step to get mwz (while using pcntl) to take into account the limits of
'delivery server quotas', 'customer quotas', and 'change server at'?
If you have a patch, I will get you test results ;)
Is there anything else I can do to make this work?

@Mxyzptlk FishStix @bidorbuy @pradeep sharma @Rob @Vpul Shah @VVT
What is your solution to the above problems?

The only thing I can think of is to (while pcntl is enabled) to use a single unlimited smtp, set unlimited quota, and do away with change server at, but that was not the original idea...as (some of the) power & beauty of mwz is in the quota controls and sending options...
 
Last edited:
I just disabled it. An ESP cannot make use of this feature since the quota allocation doesn't work properly. But for a regular user, it doesn't matter in most cases and the resources can be squeezed out using PCNTL for max performance. The old fashion send-parallel command worked better though !
 
@VVT
Yes, agreed, it's not easily fixed by adjusted settings. Perhaps the lowest denominator among the DS' (so a single cron run cannot break any quota) will allow using pcntl to some degree. For the send-parallel-campaigns command one might need to put up a dedicated kill after campaigns are finished (disregarding off-quotas). Thx for your thoughts, I wonder how to help fix it besides test results.

@twisted1919
OK, so the next step is adjusted code to use shmop/etc and then testing. As this is an open issue since a long time and everybody would benefit, could you please make it a top priority now? If you have a patch, I'll get you test results asap! Is there anything else I can do to make this work?
 
frm.mwz said:
@VVT
Yes, agreed, it's not easily fixed by adjusted settings. Perhaps the lowest denominator among the DS' (so a single cron run cannot break any quota) will allow using pcntl to some degree. For the send-parallel-campaigns command one might need to put up a dedicated kill after campaigns are finished (disregarding off-quotas). Thx for your thoughts, I wonder how to help fix it besides test results.
Click to expand...

Is that so ? I have been using that for months and never had to do anything from command line:rolleyes:. Then I stopped using it after seeing that the support was withdrawn starting from some release. One more thing I liked is, campaign will be sent immediately. It works like a daemon than an init script triggered by a cron. Though, I'm not against PCNTL , if it worked well with quota settings.
 
@VVT
Great, thx for the tip, I only tested it for a bit, with good speeds (couple thou a min) and then had to kill, as it would also go over quotas, and then there was pcntl (with up to 50!). The wonderful thing with mwz is that there are various options to achieve one thing, adjustable to one's circumstances...just with pcntl I still have to find the best way.

@twisted1919
Here is an idea. As of now, pcntl will take campaigns and send in parallel processes using any available/assigned DS (and in cli verbose one can see it is using them one by one but not by quota).
Would it be possible, and perhaps very desirable, to have the option to set for each campaign, if to send parallel, and then also which DS, but then additionally specify how many processes per DS (e.g. single), and yet have e.g. ten DS send in parallel (each DS only one process though) this way (avoiding overload by multiple sending per DS, but achieving super high speed)?
 
Last edited:
frm.mwz said:
Would it be possible, and perhaps very desirable, to have the option to set for each campaign, if to send parallel, and then also which DS, but then additionally specify how many processes per DS (e.g. single), and yet have e.g. ten DS send in parallel (each DS only one process though) this way (avoiding overload by multiple sending per DS, but achieving super high speed)?
Click to expand...
Unfortunately this workflow is not really possible given the way sending campaigns work.
However, we will get a fix for pcntl too.
 
You must log in or register to reply here.
Back
Top