Mailing List Archive

Synching a FRITZ!Box with DAViCal
Hi,

As of release 07.21 of the FRITZ!Box operating system, it supports external
addressbooks via CardDAV. So I tried...

TL;DR: DAViCal responds to an initial PROPFIND with characteristics of the
address book. A Nextcloud CardDAV server (with the same content as
the DAViCal server) responds to (assumingly) the same PROPFIND with address
book characteristics AND the full list of entries (IDs) in the address book.

Details:

After some tweaking to get the communication going, this is what I see in
the logs:

"PROPFIND /caldav.php/jv/addresses/ HTTP/1.1" 207 325

That is looking good but it's a small response. What I see next is:

"PROPFIND /caldav.php/jv/ HTTP/1.1" 207 359

So it queries the user info. It gets information about all calendars,
addressbooks, and more. What follows is even more surprising:

"PROPFIND /caldav.php/jv/music/ HTTP/1.1" 207 106900

Now it has selected one of the calendars (music) and queries it. Then it
continues with a series of REPORTS, all for the music calendar:

"REPORT /caldav.php/jv/music/ HTTP/1.1" 207 9812
"REPORT /caldav.php/jv/music/ HTTP/1.1" 207 9672
...some more...
"REPORT /caldav.php/jv/music/ HTTP/1.1" 207 9140
"REPORT /caldav.php/jv/music/ HTTP/1.1" 207 3526

When this is completed the FRITZOS_CardDAV_Client stops (finishes?) and the
web UI reports that "The update was successful". However, the address book
is (still) empty. Not surprising since there are no contacts in the music
calendar (or any other calendar).

When I do the same with a Nextcloud CardDAV server, the sequence of
requests is:

"PROPFIND /remote.php/dav/addressbooks/users/jv/addresses/ HTTP/1.1" 207 79137
"REPORT /remote.php/dav/addressbooks/users/jv/addresses/ HTTP/1.1" 207 504762
"REPORT /remote.php/dav/addressbooks/users/jv/addresses/ HTTP/1.1" 207 441377
"REPORT /remote.php/dav/addressbooks/users/jv/addresses/ HTTP/1.1" 207 203985
"REPORT /remote.php/dav/addressbooks/users/jv/addresses/ HTTP/1.1" 207 264726
"REPORT /remote.php/dav/addressbooks/users/jv/addresses/ HTTP/1.1" 207 363332
"REPORT /remote.php/dav/addressbooks/users/jv/addresses/ HTTP/1.1" 207 368285
"REPORT /remote.php/dav/addressbooks/users/jv/addresses/ HTTP/1.1" 207 366294
"REPORT /remote.php/dav/addressbooks/users/jv/addresses/ HTTP/1.1" 207 687129
"REPORT /remote.php/dav/addressbooks/users/jv/addresses/ HTTP/1.1" 207 379307

Note that the first PROPFIND returns a lot of information, it includes all
entry IDs in the addresses. Upon completion the address book on the
FRITZ!Box is filled with the correct information.

Any idea what is going on?

-- Johan





_______________________________________________
Davical-general mailing list
Davical-general@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/davical-general
Re: Synching a FRITZ!Box with DAViCal [ In reply to ]
The requests are genenated by the CardDAV client software (which one do you use?) based on the interpretation of previous reponses it got... so likely to be a client side issue (config/bug)... it shouldn't look in a calendar ressource for contacts in the first place I believe...

You could compare request and response between:

"PROPFIND /caldav.php/jv/addresses/ HTTP/1.1" 207 325

and

"PROPFIND /remote.php/dav/addressbooks/users/jv/addresses/ HTTP/1.1" 207 79137

The server would only answer what it is requested for...

In the first case something causes the client to request user info:

"PROPFIND /caldav.php/jv/ HTTP/1.1" 207 359

then the client request:

"PROPFIND /caldav.php/jv/music/ HTTP/1.1" 207 106900

Which it shouldn't as it is a calendar... not an addressbook... (assuming here music is set to be a calendar ressource in your Davical environment ;-)

Have you tried with different CardDAV client software to compare behavior between clients, see if you get same or similar results?

You should be able to access request/response via client software logging or server side logging via Davical... by tweaking log settings...

Accessing request and response content may help to give a clue perhaps...

On 29 October 2020 14:55:14 CET, Johan Vromans <jvromans@squirrel.nl> wrote:
>Hi,
>
>As of release 07.21 of the FRITZ!Box operating system, it supports
>external
>addressbooks via CardDAV. So I tried...
>
>TL;DR: DAViCal responds to an initial PROPFIND with characteristics of
>the
>address book. A Nextcloud CardDAV server (with the same content as
>the DAViCal server) responds to (assumingly) the same PROPFIND with
>address
>book characteristics AND the full list of entries (IDs) in the address
>book.
>
>Details:
>
>After some tweaking to get the communication going, this is what I see
>in
>the logs:
>
>"PROPFIND /caldav.php/jv/addresses/ HTTP/1.1" 207 325
>
>That is looking good but it's a small response. What I see next is:
>
>"PROPFIND /caldav.php/jv/ HTTP/1.1" 207 359
>
>So it queries the user info. It gets information about all calendars,
>addressbooks, and more. What follows is even more surprising:
>
>"PROPFIND /caldav.php/jv/music/ HTTP/1.1" 207 106900
>
>Now it has selected one of the calendars (music) and queries it. Then
>it
>continues with a series of REPORTS, all for the music calendar:
>
>"REPORT /caldav.php/jv/music/ HTTP/1.1" 207 9812
>"REPORT /caldav.php/jv/music/ HTTP/1.1" 207 9672
>...some more...
>"REPORT /caldav.php/jv/music/ HTTP/1.1" 207 9140
>"REPORT /caldav.php/jv/music/ HTTP/1.1" 207 3526
>
>When this is completed the FRITZOS_CardDAV_Client stops (finishes?) and
>the
>web UI reports that "The update was successful". However, the address
>book
>is (still) empty. Not surprising since there are no contacts in the
>music
>calendar (or any other calendar).
>
>When I do the same with a Nextcloud CardDAV server, the sequence of
>requests is:
>
>"PROPFIND /remote.php/dav/addressbooks/users/jv/addresses/ HTTP/1.1"
>207 79137
>"REPORT /remote.php/dav/addressbooks/users/jv/addresses/ HTTP/1.1" 207
>504762
>"REPORT /remote.php/dav/addressbooks/users/jv/addresses/ HTTP/1.1" 207
>441377
>"REPORT /remote.php/dav/addressbooks/users/jv/addresses/ HTTP/1.1" 207
>203985
>"REPORT /remote.php/dav/addressbooks/users/jv/addresses/ HTTP/1.1" 207
>264726
>"REPORT /remote.php/dav/addressbooks/users/jv/addresses/ HTTP/1.1" 207
>363332
>"REPORT /remote.php/dav/addressbooks/users/jv/addresses/ HTTP/1.1" 207
>368285
>"REPORT /remote.php/dav/addressbooks/users/jv/addresses/ HTTP/1.1" 207
>366294
>"REPORT /remote.php/dav/addressbooks/users/jv/addresses/ HTTP/1.1" 207
>687129
>"REPORT /remote.php/dav/addressbooks/users/jv/addresses/ HTTP/1.1" 207
>379307
>
>Note that the first PROPFIND returns a lot of information, it includes
>all
>entry IDs in the addresses. Upon completion the address book on the
>FRITZ!Box is filled with the correct information.
>
>Any idea what is going on?
>
>-- Johan
>
>
>
>
>
>_______________________________________________
>Davical-general mailing list
>Davical-general@lists.sourceforge.net
>https://lists.sourceforge.net/lists/listinfo/davical-general

--
Sent from my Android device with K-9 Mail. Please excuse my brevity.


_______________________________________________
Davical-general mailing list
Davical-general@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/davical-general
Re: Synching a FRITZ!Box with DAViCal [ In reply to ]
On Sat, 31 Oct 2020 10:58:54 +0100, Richard Leger <richard.leger@gmail.com>
wrote:

> You should be able to access request/response via client software logging
> or server side logging via Davical... by tweaking log settings...

The client software is FRITZOS_CardDAV_Client which is a built-in function
of the FRITZ!Box. There is nothing I can do to get debugging info or
whatsoever from the client, the FRITZ!Box is a closed system.

I added debug logging to the DAViCal config:

$c->dbg["request"] = 1; // The request headers & content
$c->dbg['response'] = 1; // The response headers & content
$c->dbg_filter["remoteIP"][] = '192.168.1.1';

Where 192.168.1.1 is the IP address of the FRITZ!Box. This gives some
insight but unfortunately nginx logging truncates the messages.

At least the requests are clear so I can replay these manually.

Request #1 is a PROPFIND for /caldav.php/jv/addresses/ with content

<d:propfind xmlns:d="DAV:">
<d:prop>
<d:current-user-principal/>
</d:prop>
</d:propfind>

The response is:

<?xml version="1.0" encoding="utf-8" ?>
<multistatus xmlns="DAV:">
<response>
<href>/caldav.php/jv/addresses/</href>
<propstat>
<prop>
<current-user-principal>
<href>/caldav.php/jv/</href>
</current-user-principal>
</prop>
<status>HTTP/1.1 200 OK</status>
</propstat>
</response>
</multistatus>

AFAIK this looks okay.

However when I issue the same request at Nextcloud (SabreDAV), the
response is:

<?xml version="1.0" encoding="utf-8" ?>
<d:multistatus xmlns:d="DAV:" xmlns:s="http://sabredav.org/ns" xmlns:card="urn:ietf:params:xml:ns:carddav" xmlns:oc="http://owncloud.org/ns" xmlns:nc="http://nextcloud.org/ns">
<d:response>
<d:href>/remote.php/dav/addressbooks/users/jv/addresses/</d:href>
<d:propstat>
<d:prop>
<d:current-user-principal>
<d:href>/remote.php/dav/principals/users/jv/</d:href>
</d:current-user-principal>
</d:prop>
<d:status>HTTP/1.1 200 OK</d:status>
</d:propstat>
</d:response>
<d:response>
<d:href>/remote.php/dav/addressbooks/users/jv/addresses/0c6d5jhm-8q2l-uioo-nadl-naymc80iqu9h.vcf</d:href>
<d:propstat>
<d:prop>
<d:current-user-principal>
<d:href>/remote.php/dav/principals/users/jv/</d:href>
</d:current-user-principal>
</d:prop>
<d:status>HTTP/1.1 200 OK</d:status>
</d:propstat>
</d:response>
<d:response>
<d:href>/remote.php/dav/addressbooks/users/jv/addresses/0dkhaxj9-66oc-p0x5-uchs-cehwk5hgqi6c.vcf</d:href>
<d:propstat>
<d:prop>
<d:current-user-principal>
<d:href>/remote.php/dav/principals/users/jv/</d:href>
</d:current-user-principal>
</d:prop>
<d:status>HTTP/1.1 200 OK</d:status>
</d:propstat>
</d:response>

...etc...

<d:href>/remote.php/dav/addressbooks/users/jv/addresses/e1d4b9cb-b1b7-4c38-9a87-bf7e2d9db4ab.vcf</d:href>
<d:propstat>
<d:prop>
<d:current-user-principal>
<d:href>/remote.php/dav/principals/users/jv/</d:href>
</d:current-user-principal>
</d:prop>
<d:status>HTTP/1.1 200 OK</d:status>
</d:propstat>
</d:response>
</d:multistatus>

So a major difference is that Nextcloud reports the requested
current-user-principal AND ALSO the hrefs of ALL of the addressbook entries.

I'm not sufficiently familiar with the DAV protocols to tell if either
is wrong or right.

Anyway, let's proceed with DAViCal.

Request #2 PROPFIND /caldav.php/jv/

<propfind xmlns="DAV:">
<prop>
<addressbook-home-set xmlns="urn:ietf:params:xml:ns:carddav"/>
</prop>
</propfind>

Response #2:

<?xml version="1.0" encoding="utf-8" ?>
<multistatus xmlns="DAV:" xmlns:VC="urn:ietf:params:xml:ns:carddav">
<response>
<href>/caldav.php/jv/</href>
<propstat>
<prop>
<VC:addressbook-home-set>
<href>/caldav.php/jv/</href>
</VC:addressbook-home-set>
</prop>
<status>HTTP/1.1 200 OK</status>
</propstat>
</response>
</multistatus>

Again, this looks okay to me.

Request #3 PROPFIND /caldav.php/jv/

<propfind xmlns="DAV:">
<prop>
<displayname />
<resourcetype />
</prop>
</propfind>

When I perform this request manually, using curl, I get
Response #3:

<?xml version="1.0" encoding="utf-8" ?>
<multistatus xmlns="DAV:">
<response>
<href>/caldav.php/jv/</href>
<propstat>
<prop>
<displayname>Johan Vromans</displayname>
<resourcetype>
<collection/>
<principal/>
</resourcetype>
</prop>
<status>HTTP/1.1 200 OK</status>
</propstat>
</response>
</multistatus>

But when the FRITZOS_CardDAV_Client performs this query, it gets a
different response. The first response is the same, but it is followed
by a series of responses for the addressbook and all calendars.

<?xml version="1.0" encoding="utf-8" ?>
<multistatus xmlns="DAV:" xmlns:C="urn:ietf:params:xml:ns:caldav" xmlns:C1="urn:ietf:params:xml:ns:carddav" xmlns:C2="http://calendarserver.org/ns/">
<response>
<href>/caldav.php/jv/</href>
<propstat>
<prop>
<displayname>Johan Vromans</displayname>
<resourcetype>

--- here the logging is truncated, but seems to continue in another log message ---

<collection/>
<C1:addressbook/>
</resourcetype>
</prop>
<status>HTTP/1.1 200 OK</status>
</propstat>
</response>
<response>
<href>/caldav.php/jv/try/</href>
<propstat>
<prop>
<displayname>Testing</displayname>
<resourcetype>
<collection/>
<C:calendar/>
</resourcetype>
</prop>
<status>HTTP/1.1 200 OK</status>
</propstat>
</response>
... and <response/> entries for all other calendars

I cannot figure out why the same request would result in different
replies. According to the debug information from the DAViCal server
the message headers are the same as the ones that curl sends.

Server:

:***************** Request Header ****************
:PROPFIND /caldav.php/jv/
headers:-->Connection: close
headers:-->Host: davical.squirrel.nl
headers:-->Depth: 1
headers:-->User-Agent: FRITZOS_CardDAV_Client/1.0
headers:-->Content-Type: application/xml; charset=UTF-8
headers:-->Content-Length: 78
headers:-->Authorization: Delicious tasty password eaten by debugging monster!
:******************** Request ********************
request:--><propfind xmlns="DAV:"><prop><displayname /><resourcetype /></prop></propfind>
:Response status 207 for PROPFIND /caldav.php/jv/
:***************** Response Header ****************
headers:-->Server: 1.1
headers:-->DAV: 1, 2, 3, access-control, calendar-access, calendar-schedule
headers:-->DAV: extended-mkcol, bind, addressbook, calendar-auto-schedule, calendar-proxy
headers:-->ETag: "1433f10e332adeaf28c202b17bc2b144"
headers:-->X-DAViCal-Version: DAViCal/1.1.8; DB/1.3.3
headers:-->Content-type: text/xml; charset="utf-8"
:******************** Response ********************
response:--><?xml version="1.0" encoding="utf-8" ?>
response:--><multistatus xmlns="DAV:" xmlns:C="urn:ietf:params:xml:ns:caldav" xmlns:C1="urn:ietf:params:xml:ns:carddav" xmlns:C2="http://calendarserver.org/ns/">
response:--> <response>
response:--> <href>/caldav.php/jv/</href>
response:--> <propstat>
response:--> <prop>
response:--> <displayname>Johan Vromans</displayname>
response:--> <resourcetype>

Curl:

$ cat req03.xml
<propfind xmlns="DAV:"><prop><displayname /><resourcetype /></prop></propfind>
$ curl --user ... -v -s -X PROPFIND --data-binary @req03.xml https://davical.squirrel.nl/caldav.php/jv/
* Trying 192.168.1.13:443...
* TCP_NODELAY set
* Connected to davical.squirrel.nl (192.168.1.13) port 443 (#0)
* ALPN, offering h2
* ALPN, offering http/1.1
* ...
* SSL certificate verify ok.
* Using HTTP2, server supports multi-use
* Connection state changed (HTTP/2 confirmed)
* Copying HTTP/2 data in stream buffer to connection buffer after upgrade: len=0
* Server auth using Basic with user 'jv'
* Using Stream ID: 1 (easy handle 0x55e96bbba9c0)
> PROPFIND /caldav.php/jv/ HTTP/2
> Host: davical.squirrel.nl
> Authorization: Basic xxxxxxxxxxxxxxxx
> User-Agent: curl/7.66.0
> Accept: */*
> Content-Length: 78
> Content-Type: application/x-www-form-urlencoded
>
* Connection state changed (MAX_CONCURRENT_STREAMS == 128)!
* We are completely uploaded and fine
< HTTP/2 207
< date: Sun, 01 Nov 2020 14:29:48 GMT
< content-type: text/xml; charset="utf-8"
< content-length: 343
< server: 1.1
< dav: 1, 2, 3, access-control, calendar-access, calendar-schedule
< dav: extended-mkcol, bind, addressbook, calendar-auto-schedule, calendar-proxy
< etag: "da9dffa30f5e1e568a31145fd695b883"
< x-davical-version: DAViCal/1.1.8; DB/1.3.3
<
<?xml version="1.0" encoding="utf-8" ?>
<multistatus xmlns="DAV:">
<response>
<href>/caldav.php/jv/</href>
<propstat>
<prop>
<displayname>Johan Vromans</displayname>
<resourcetype>
<collection/>
<principal/>
</resourcetype>
</prop>
<status>HTTP/1.1 200 OK</status>
</propstat>
</response>
</multistatus>
* Connection #0 to host davical.squirrel.nl left intact

I've no clue... do you?

-- Johan


_______________________________________________
Davical-general mailing list
Davical-general@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/davical-general
Re: Synching a FRITZ!Box with DAViCal [ In reply to ]
Hi Johan,

DAViCal is returning different information because of the Depth HTTP
header.

Depth:0 = return information about the item you're querying (so just
the Principal - you)
Depth:1 = return information about the collections under the item
(calendars, addressbooks)
Depth:2 = return information about what is in the collections

You can experiment with this by adding " -H Depth:1" (etc) to your curl
command. And this is why the result you get running curl differs from
what the FRITZ!Box is being sent.

Interestingly, I have found someone complaining that Nextcloud ignores
the depth setting:
https://help.nextcloud.com/t/problems-with-search-method-via-webdav/45102

Cheers,
Andrew

--
Andrew Ruthven, Wellington, New Zealand
andrew@etc.gen.nz |
Catalyst Cloud: | This space intentionally left blank
https://catalystcloud.nz |



_______________________________________________
Davical-general mailing list
Davical-general@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/davical-general