Problem with auto-completion of custom fields

Groundarker

New Member
Dear Mailwizz staff,
I come back to you to ask you some explanations about the synchronization function of the custom fields.

After the request that I have expressed to you here: https://forum.mailwizz.com/threads/auto-completion-of-custom-fields-created-for-a-list.6632/
I kindly asked you to insert TAGs to auto-fill in the last email opening date, last click opening date, last sent date, etc.

I thank you for the implementation and it seems that these TAGs work.

The problem I objectively encounter now is this:
I have 2 lists of which list n.1 is sent by setting the function of moving the user towards list n.2 in the sending schedule.

The user is correctly moved but the field last opened date, last click date, etc. remain the initial ones and the system does not update this data!

I tried to search the forum if someone else, like me, had the same problem as me but I found only one suggestion here: https://forum.mailwizz.com/threads/creating-segments-from-field-with-no-value.4206/

Here you say you can use the following cron command every 30 minutes
Code:
php -q apps/console/console.php sync-lists-custom-fields --verbose=1
and this command after running it has practically created values with apostrophes in fields where nothing appeared,

I attach screenshots below:

1590663689896.png



However even running this command the fields have not been updated! Could you please verify this problem?
Thanks as always and good job!

See you soon
 

Groundarker

New Member
@Groundarker - could you please open a support ticket with backend url and access to your app so we can run some tests on it?
Guys, I confirm that I was able to solve the problem on my own. Practically launching the daily cron manually does not create that mess as I sent you in the screenshot, while launching the cron that I indicated in the opening thread it messes up all the data.

The only problem remains the huge amount of data that the cron must process being millions of contacts and having at least about 10 custom fields or more takes a very long time, is there any way to speed up the process eventually?
Thanks a lot!
 

twisted1919

Administrator
Staff member
@Groundarker - I think we have also fixed this on our end. Also processing the custom fields will work much faster with the upcoming release.
So let us know how things work after 1.9.10 update.
 

Groundarker

New Member
Fantastic! I will definitely keep you updated on the status as soon as the new version of the software is available
 

Groundarker

New Member
Guys @twisted1919 @laurentiu , thank you for releasing the latest version of the software.
I'll be back here to inform you that I have had the opportunity to install and try it but by launching the custom field sync the waiting times are really long to autocompile the data of millions of subscribers.
Can you by chance do something to make this function more performing?

I ask you this because we (mainly) use the following fields filled in by default: SUBSCRIBER_LAST_OPEN_DATETIME, SUBSCRIBER_LAST_CLICK_DATETIME, SUBSCRIBER_LAST_SEND_DATETIME.

As you can imagine, we then segment this field in the list to be able to retrieve who has opened or clicked any newsletter in the last 30/60/90/120 days. If updating the custom fields takes a long time in updating the above fields, the segment will consequently lose many contacts that could actually fall within the range of days.

You have certainly improved your performance with the latest update, but with millions of contacts under management the process takes a long time to process! I really hope you will find another alternative solution for this problem ...

As always good work and congratulations for this fantastic project!
 

twisted1919

Administrator
Staff member
@Groundarker - Thank you for looking into this, i can't tell you how much this feedback is appreciated ;)
While we're super busy with other tasks, i'll make sure we will improve this one for the next version, i think we can implement parallel processing via pcntl like we do for lots of heavy commands ;)
I will keep you posted.
 

twisted1919

Administrator
Staff member
@Groundarker - following up with this, here's the speed from your version, that is for 100k subscribers and updating a custom field:
Code:
[2020-06-22 10:12:52] - Batch is done!
[2020-06-22 10:12:52] - Loading subscribers set for the list with limit: 1000 and offset 100000
[2020-06-22 10:12:52] - Done, no more subscribers for this list!
[2020-06-22 10:12:52] - Done!
docker/php/runcmd.sh sync-lists-custom-fields --verbose=1  
0.14s user 0.10s system 0% cpu 
14:26.00 total
And this is the improvement:
Code:
[2020-06-22 09:56:33] - Batch is done!
[2020-06-22 09:56:34] - Loading subscribers set for the list with limit: 1000 and offset 100000
[2020-06-22 09:56:34] - Done, no more subscribers for this list!
[2020-06-22 09:56:34] - Done!
docker/php/runcmd.sh sync-lists-custom-fields --verbose=1  
0.09s user 0.09s system 0% cpu 
53.330 total
So we're down from 14 minutes to under 1 minute.
We're including this in the next release.
 
Top