Mailing List Archive

Possible fix for deadlocks in database
Good afternoon,

As you know, when you are doing a massive upload to a collection, you
end up by seeing in the logs the following errors :

[Sat Apr 02 18:09:49.814518 2016] [:error] [pid 53742] [client
192.148.167.11:21670] davical: LOG:
caldav.php/egoitz@ramattack.net/addresses/eb285f17-ac04-4d04-a826-f2d63b424a65.vcf:
Query: QF: SQL error "40P01" - ERROR: deadlock detected DETALLE: Process
19727 waits for ShareLock on transaction 69330601; blocked by process
19728. Process 19728 waits for ExclusiveLock on tuple (0,29) of relation
1206940 of database 1206795; blocke
.
.
.
.
.
.

[Sat Apr 02 18:09:49.818981 2016] [:error] [pid 53742] [client
192.148.167.11:21670] davical: LOG:
caldav.php/egoitz@ramattack.net/addresses/eb285f17-ac04-4d04-a826-f2d63b424a65.vcf:
Query: QF: SQL error "25P02" - ERROR: current transaction is aborted,
commands ignored until end of transaction block"

Basically this seems to be because perhaps Davical was designed having
in mind all clients behave uploading or deleting for instance...
elements but, one by one.... but nowadays... some clients
work this way, and other ones, don't. You can easily adapt Sogo
connector for behaving this proper way (and fix some other bug too by
the way. You can download this patched by me version from
https://www.saremail.com/docs/sogosarenet), but not for intance Apple
contacts app (as it's not open source). In "El Capitan" it seems they
have modified the CardDAV code by openning several
simultaneous connectionsto the CardDAV server for faster uploads.
Happens the same (although in CalDAV) with Lightning, Calendar app in
Mac... and perhaps some other too....

I have search in Google for a fix for this problem, but have not been
able to find... just some ideas of Andrew McMillan saying he was
thinking on writting something for fix the issue using memcached
and so... After reading that, I stayed thinking in it... and think have
found a simplier solution to this problem (well, at least, simplier than
using memcached for me). I wanted to share the following patch
for Davical 1.1.4 with you in order to know your opinion or contributing
it. I know things could be perhaps better written but, this patch seems
to work fine and could very easily be improved for working
even slightly better (although have not been able to make it fail, all
massive uploads and removals have succeeded) if needed.

The patch is attached to this mail. Let me know your impressions about it :)

Best regards,



sarenet
*Egoitz Aurrekoetxea*
Departamento de sistemas
944 209 470
Parque Tecnológico. Edificio 103
48170 Zamudio (Bizkaia)
egoitz@sarenet.es <mailto:egoitz@sarenet.es>
www.sarenet.es <http://www.sarenet.es>

Antes de imprimir este correo electrónico piense si es necesario hacerlo.
Possible fix for deadlocks in database [ In reply to ]
Good afternoon,

As you know, when you are doing a massive upload to a collection, you
end up by seeing in the logs the following errors :

[Sat Apr 02 18:09:49.814518 2016] [:error] [pid 53742] [client
192.148.167.11:21670] davical: LOG:
caldav.php/egoitz@ramattack.net/addresses/eb285f17-ac04-4d04-a826-f2d63b424a65.vcf:
Query: QF: SQL error "40P01" - ERROR: deadlock detected DETALLE: Process
19727 waits for ShareLock on transaction 69330601; blocked by process
19728. Process 19728 waits for ExclusiveLock on tuple (0,29) of relation
1206940 of database 1206795; blocke
.
.
.
.
.
.

[Sat Apr 02 18:09:49.818981 2016] [:error] [pid 53742] [client
192.148.167.11:21670] davical: LOG:
caldav.php/egoitz@ramattack.net/addresses/eb285f17-ac04-4d04-a826-f2d63b424a65.vcf:
Query: QF: SQL error "25P02" - ERROR: current transaction is aborted,
commands ignored until end of transaction block"

Basically this seems to be because perhaps Davical was designed having
in mind all clients behave uploading or deleting for instance...
elements but, one by one.... but nowadays... some clients
work this way, and other ones, don't. You can easily adapt Sogo
connector for behaving this proper way (and fix some other bug too by
the way. You can download this patched by me version from
https://www.saremail.com/docs/sogosarenet), but not for intance Apple
contacts app (as it's not open source). In "El Capitan" it seems they
have modified the CardDAV code by openning several
simultaneous connectionsto the CardDAV server for faster uploads.
Happens the same (although in CalDAV) with Lightning, Calendar app in
Mac... and perhaps some other too....

I have search in Google for a fix for this problem, but have not been
able to find... just some ideas of Andrew McMillan saying he was
thinking on writting something for fix the issue using memcached
and so... After reading that, I stayed thinking in it... and think have
found a simplier solution to this problem (well, at least, simplier than
using memcached for me). I wanted to share the following patch
for Davical 1.1.4 with you in order to know your opinion or contributing
it. I know things could be perhaps better written but, this patch seems
to work fine and could very easily be improved for working
even slightly better (although have not been able to make it fail, all
massive uploads and removals have succeeded) if needed.

The patch is attached to this mail. Let me know your impressions about it :)

Best regards,


sarenet
*Egoitz Aurrekoetxea*
Departamento de sistemas
944 209 470
Parque Tecnológico. Edificio 103
48170 Zamudio (Bizkaia)
egoitz@sarenet.es <mailto:egoitz@sarenet.es>
www.sarenet.es <http://www.sarenet.es>

Antes de imprimir este correo electrónico piense si es necesario hacerlo.