Mailing List Archive

migrating/importing larger calendar files
Hi,

I am struggling with moving an running calendar file from DAVICal 1.1.1
to a newer installation on my privately operating home server (DAVICal
v1.1.7 on ubuntu0.18.04.6 w/ PHP Version 7.2.24-0. The point is I only
want to transfer the calendar file, not the whole database. None of the
explained and described ways seem to work properly. Also discussions in
this mailing list do not seem to deliver a solution.

The situation is the following: from the old DAVICal server I can backup
the calendar file of a specific user (it's actually the "family
calendar"...) via the described way under
http://wiki.davical.org/index.php/Backups#Per_user_ics.2Fvcard_export with

" psql davical -Atc "SELECT array_to_string(array(SELECT caldav_data FROM caldav_data WHERE dav_name LIKE '/family/calendar/%'),'');" > family.ics"

This delivers a file with basically ~4900 events and 4.7M in size. I can
read the file and all events seems to be ok and the file seems not to
corrupted.

Importing this file via the procedure described in
http://wiki.davical.org/index.php/How_can_I_migrate_to_DAViCal_using_already_generated_.ics_files%3F#Importing_.ics_Files_into_DAViCal_using_the_import_tool
fails. A collection for the user will be set up but the collection
remains empty.


I played around a lot and tested different ways. The only working
approach I could find is the following:

when setting up a collection for the user and open it from the admin
menu (web console) there is the feature to load from file and add items
to the collection and append to existing events. This does not work with
the original 4.7M ics file mentioned above. When splitting the large ics
backup file via " csplit -n 4 family.ics /BEGIN:VCALENDAR/ '{*}' " into
the ~4900 single event ics files and importing them via the web console
and appending the file this seems to work (I have tested this with a few
files only...) - so the question is: how to do that efficiently with
several thousand files ? What would be the corresponding console command
to do this via a script instead of clicking the buttons in the web console ?


Is there a general problem with importing files exceeding a certain size
? Another phenomena was that when importing address books from different
users this seemed to work with vcf-files below 1M but for larger files
the contacts could not be imported ?

vcf file 1     89 contacts    868K    -> works
vcf file 2    161 contacts   2.9M    -> does not work
vcf file 3    379 contacts   3.3M    -> does not work

The vcf files where generated from the old DAVICal server in a similar
way as described above
http://wiki.davical.org/index.php/Backups#Per_user_ics.2Fvcard_export ...


Much thanks for supporting and best regards,
Timm
Re: migrating/importing larger calendar files [ In reply to ]
Hi,

On 2020-08-02 01:17, Timm Waterstradt wrote:
> I am struggling with moving an running calendar file from DAVICal 1.1.1
> to a newer installation
[…Importing fails…]
>
> Is there a general problem with importing files exceeding a certain size ?
> Another phenomena was that when importing address books from different
> users this seemed to work with vcf-files below 1M but for larger files the
> contacts could not be imported ?
>
> vcf file 1     89 contacts    868K    -> works
> vcf file 2    161 contacts   2.9M    -> does not work
> vcf file 3    379 contacts   3.3M    -> does not work

I don't know too much about the DAVical internals and what could possibly
go wrong - but looking at the file sizes, I see that every of your uploads
over 2M fails. This is the default maximum upload size for PHP.

Check the upload_max_filesize and post_max_size options.

Also the other options mentioned here:
https://www.php.net/manual/en/features.file-upload.common-pitfalls.php


Thanks,
Adalbert


_______________________________________________
Davical-general mailing list
Davical-general@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/davical-general
Re: migrating/importing larger calendar files [ In reply to ]
Hi, much thanks for the reply. This is a good hint. I will check and retry. I suppose the mentioned 'switches' are to be set in the php.ini?

Best regards,
Timm

Am 3. August 2020 09:35:44 MESZ schrieb Adalbert Michelic <am+davical@timeanddate.com>:
>Hi,
>
>On 2020-08-02 01:17, Timm Waterstradt wrote:
>> I am struggling with moving an running calendar file from DAVICal
>1.1.1
>> to a newer installation
>[…Importing fails…]
>>
>> Is there a general problem with importing files exceeding a certain
>size ?
>> Another phenomena was that when importing address books from
>different
>> users this seemed to work with vcf-files below 1M but for larger
>files the
>> contacts could not be imported ?
>>
>> vcf file 1     89 contacts    868K    -> works
>> vcf file 2    161 contacts   2.9M    -> does not work
>> vcf file 3    379 contacts   3.3M    -> does not work
>
>I don't know too much about the DAVical internals and what could
>possibly
>go wrong - but looking at the file sizes, I see that every of your
>uploads
>over 2M fails. This is the default maximum upload size for PHP.
>
>Check the upload_max_filesize and post_max_size options.
>
>Also the other options mentioned here:
>https://www.php.net/manual/en/features.file-upload.common-pitfalls.php
>
>
>Thanks,
>Adalbert
>
>
>_______________________________________________
>Davical-general mailing list
>Davical-general@lists.sourceforge.net
>https://lists.sourceforge.net/lists/listinfo/davical-general

--
TW mobil mit K-9 Mail gesendet.
Re: migrating/importing larger calendar files [ In reply to ]
Hi Timm,

yes, please note that there might be several php.ini on your system depending on your type of OS and the number of installed „slots“ of PHP.

My Debian:
/etc/php/7.0/apache2/php.ini
/etc/php/7.0/cli/php.ini
/etc/php/7.3/apache2/php.ini
/etc/php/7.3/cli/php.ini

or Gentoo:
/etc/php/apache2-php7.3/php.ini
/etc/php/apache2-php7.4/php.ini
/etc/php/cli-php7.3/php.ini
/etc/php/cli-php7.4/php.ini


Take care

Tino

> Am 03.08.2020 um 10:01 schrieb Timm Waterstradt <timm.waterstradt@web.de>:
>
> Hi, much thanks for the reply. This is a good hint. I will check and retry. I suppose the mentioned 'switches' are to be set in the php.ini?
>
> Best regards,
> Timm
>
> Am 3. August 2020 09:35:44 MESZ schrieb Adalbert Michelic <am+davical@timeanddate.com>:
> Hi,
>
> On 2020-08-02 01:17, Timm Waterstradt wrote:
> I am struggling with moving an running calendar file from DAVICal 1.1.1
> to a newer installation
> […Importing fails…]
> >
> Is there a general problem with importing files exceeding a certain size ?
> Another phenomena was that when importing address books from different
> users this seemed to work with vcf-files below 1M but for larger files the
> contacts could not be imported ?
>
> vcf file 1 89 contacts 868K -> works
> vcf file 2 161 contacts 2.9M -> does not work
> vcf file 3 379 contacts 3.3M -> does not work
>
> I don't know too much about the DAVical internals and what could possibly
> go wrong - but looking at the file sizes, I see that every of your uploads
> over 2M fails. This is the default maximum upload size for PHP.
>
> Check the upload_max_filesize and post_max_size options.
>
> Also the other options mentioned here:
> https://www.php.net/manual/en/features.file-upload.common-pitfalls.php
>
>
> Thanks,
> Adalbert
> Davical-general mailing list
> Davical-general@lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/davical-general
>
> --
> TW mobil mit K-9 Mail gesendet.
> _______________________________________________
> Davical-general mailing list
> Davical-general@lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/davical-general



_______________________________________________
Davical-general mailing list
Davical-general@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/davical-general