Mailing List Archive: [OT] tar exclude syntax tip

[OT] tar exclude syntax tip

May 5, 2021, 6:33 AM

Post #1 of 8 (536 views)

tar version

####################################################
tar (GNU tar) 1.34
Copyright (C) 2021 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <https://gnu.org/licenses/gpl.html>.
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.

Written by John Gilmore and Jay Fenlason.
####################################################

I'm passing on this solution to help others avoid my frustration and
wasted time. If you've done "RTFM" on tar, you'll find out that "TFM"
is broken or out-of-date or whatever, re: "--exclude=PATTERN". I'm
fighting the urge to turn this into a rant. Here's my situation...

I either log in as root or "su -" and then "cd /home". I want to tar
up /home/waltdnes, and transfer it to another machine. While I'm at it,
I want to exlude directory /home/waltdnes/.cache/ and all *.xz files in
directory /home/waltdnes/pm/ The "--exclude=" never worked. After much
hair pulling, I was ready to give up on the exclude, and simply transfer
all the unnecessary garbage.

Then "I asked Mr. Google". It seems that I wasn't the only person
running into problems. After some searching, I finally found a syntax
that works...

####################################################
#!/bin/bash
export GZIP=-9
tar cvzf wd.tgz --exclude ".cache/*" --exclude "pm/*.xz" waltdnes
####################################################

Notes...

1) This is obviously not in line with the man page. Specifically,
"--exclude" is followed by one space, not an equals sign.

2) ***THERE MUST BE EXACTLY ONE SPACE BETWEEN EACH WORD***

3) All directories and/or files to exclude must be listed as relative
paths to the directory being tarred, i.e. last parameter on the command
line.

4) I don't know the maximum line-length, which would limit the number of
--exclude entries. In those cases, I wonder if "--exclude-from=FILE"
works as "--exclude-from FILE".

--
Walter Dnes <waltdnes@waltdnes.org>
I don't run "desktop environments"; I run useful applications

Re: [OT] tar exclude syntax tip [ In reply to ]

gentoo at tastytea

May 5, 2021, 7:03 AM

Post #2 of 8 (536 views)

Permalink

On 2021-05-05 09:33-0400 "Walter Dnes" <waltdnes@waltdnes.org> wrote:

> tar version
>
> ####################################################
> tar (GNU tar) 1.34
> Copyright (C) 2021 Free Software Foundation, Inc.
> License GPLv3+: GNU GPL version 3 or later
> <https://gnu.org/licenses/gpl.html>. This is free software: you are
> free to change and redistribute it. There is NO WARRANTY, to the
> extent permitted by law.
>
> Written by John Gilmore and Jay Fenlason.
> ####################################################
>
> I'm passing on this solution to help others avoid my frustration and
> wasted time. If you've done "RTFM" on tar, you'll find out that "TFM"
> is broken or out-of-date or whatever, re: "--exclude=PATTERN". I'm
> fighting the urge to turn this into a rant. Here's my situation...
>
> I either log in as root or "su -" and then "cd /home". I want to
> tar up /home/waltdnes, and transfer it to another machine. While I'm
> at it, I want to exlude directory /home/waltdnes/.cache/ and all *.xz
> files in directory /home/waltdnes/pm/ The "--exclude=" never worked.
> After much hair pulling, I was ready to give up on the exclude, and
> simply transfer all the unnecessary garbage.
>
> Then "I asked Mr. Google". It seems that I wasn't the only person
> running into problems. After some searching, I finally found a syntax
> that works...
>
> ####################################################
> #!/bin/bash
> export GZIP=-9
> tar cvzf wd.tgz --exclude ".cache/*" --exclude "pm/*.xz" waltdnes
> ####################################################
>
> Notes...
>
> 1) This is obviously not in line with the man page. Specifically,
> "--exclude" is followed by one space, not an equals sign.
>
> 2) ***THERE MUST BE EXACTLY ONE SPACE BETWEEN EACH WORD***
>
> 3) All directories and/or files to exclude must be listed as relative
> paths to the directory being tarred, i.e. last parameter on the
> command line.
>
> 4) I don't know the maximum line-length, which would limit the number
> of --exclude entries. In those cases, I wonder if
> "--exclude-from=FILE" works as "--exclude-from FILE".
>

This works fine here with “tar (GNU tar) 1.34”:

$ mkdir -p a/b
$ touch a/file a/b/file
$ touch a/file.xz a/b/file.xz
$ tree a
a
??? b
? ??? file
? ??? file.xz
??? file
??? file.xz

1 directory, 4 files
$ tar -cvzf test.tar.gz --exclude="a/file" --exclude="a/b/*.xz" a
a/
a/file.xz
a/b/
a/b/file
$ tar -tf test.tar.gz
a/
a/file.xz
a/b/
a/b/file

You can find out the maximum length of the command-line with
`getconf ARG_MAX`.
--
Get my PGP key with `gpg --locate-keys tastytea@tastytea.de` or at
<https://tastytea.de/tastytea.asc>.

Re: [OT] tar exclude syntax tip [ In reply to ]

nunojsilva at ist

May 5, 2021, 7:10 AM

Post #3 of 8 (536 views)

Permalink

On 2021-05-05, tastytea wrote:

> On 2021-05-05 09:33-0400 "Walter Dnes" <waltdnes@waltdnes.org> wrote:
>
>> tar version
>>
>> ####################################################
>> tar (GNU tar) 1.34
>> Copyright (C) 2021 Free Software Foundation, Inc.
>> License GPLv3+: GNU GPL version 3 or later
>> <https://gnu.org/licenses/gpl.html>. This is free software: you are
>> free to change and redistribute it. There is NO WARRANTY, to the
>> extent permitted by law.
>>
>> Written by John Gilmore and Jay Fenlason.
>> ####################################################
>>
>> I'm passing on this solution to help others avoid my frustration and
>> wasted time. If you've done "RTFM" on tar, you'll find out that "TFM"
>> is broken or out-of-date or whatever, re: "--exclude=PATTERN". I'm
>> fighting the urge to turn this into a rant. Here's my situation...
>>
>> I either log in as root or "su -" and then "cd /home". I want to
>> tar up /home/waltdnes, and transfer it to another machine. While I'm
>> at it, I want to exlude directory /home/waltdnes/.cache/ and all *.xz
>> files in directory /home/waltdnes/pm/ The "--exclude=" never worked.
>> After much hair pulling, I was ready to give up on the exclude, and
>> simply transfer all the unnecessary garbage.
>>
>> Then "I asked Mr. Google". It seems that I wasn't the only person
>> running into problems. After some searching, I finally found a syntax
>> that works...
>>
>> ####################################################
>> #!/bin/bash
>> export GZIP=-9
>> tar cvzf wd.tgz --exclude ".cache/*" --exclude "pm/*.xz" waltdnes
>> ####################################################
>>
>> Notes...
>>
>> 1) This is obviously not in line with the man page. Specifically,
>> "--exclude" is followed by one space, not an equals sign.
>>
>> 2) ***THERE MUST BE EXACTLY ONE SPACE BETWEEN EACH WORD***
>>
>> 3) All directories and/or files to exclude must be listed as relative
>> paths to the directory being tarred, i.e. last parameter on the
>> command line.
>>
>> 4) I don't know the maximum line-length, which would limit the number
>> of --exclude entries. In those cases, I wonder if
>> "--exclude-from=FILE" works as "--exclude-from FILE".
>>
>
> This works fine here with “tar (GNU tar) 1.34”:
>
> $ mkdir -p a/b
> $ touch a/file a/b/file
> $ touch a/file.xz a/b/file.xz
> $ tree a
> a
> ??? b
> ? ??? file
> ? ??? file.xz
> ??? file
> ??? file.xz
>
> 1 directory, 4 files
> $ tar -cvzf test.tar.gz --exclude="a/file" --exclude="a/b/*.xz" a
> a/
> a/file.xz
> a/b/
> a/b/file
> $ tar -tf test.tar.gz
> a/
> a/file.xz
> a/b/
> a/b/file
>
> You can find out the maximum length of the command-line with
> `getconf ARG_MAX`.

But does it work with a space instead of = as well? According to the
online manual page, it should work both ways.

--
Nuno Silva

Re: Re: [OT] tar exclude syntax tip [ In reply to ]

gentoo at tastytea

May 5, 2021, 7:23 AM

Post #4 of 8 (536 views)

Permalink

On 2021-05-05 15:10+0100 <nunojsilva@ist.utl.pt> (Nuno Silva) wrote:

> On 2021-05-05, tastytea wrote:
>
> > On 2021-05-05 09:33-0400 "Walter Dnes" <waltdnes@waltdnes.org>
> > wrote:
> >> tar version
> >>
> >> ####################################################
> >> tar (GNU tar) 1.34
> >> Copyright (C) 2021 Free Software Foundation, Inc.
> >> License GPLv3+: GNU GPL version 3 or later
> >> <https://gnu.org/licenses/gpl.html>. This is free software: you are
> >> free to change and redistribute it. There is NO WARRANTY, to the
> >> extent permitted by law.
> >>
> >> Written by John Gilmore and Jay Fenlason.
> >> ####################################################
> >>
> >> I'm passing on this solution to help others avoid my frustration
> >> and wasted time. If you've done "RTFM" on tar, you'll find out
> >> that "TFM" is broken or out-of-date or whatever, re:
> >> "--exclude=PATTERN". I'm fighting the urge to turn this into a
> >> rant. Here's my situation...
> >>
> >> I either log in as root or "su -" and then "cd /home". I want to
> >> tar up /home/waltdnes, and transfer it to another machine. While
> >> I'm at it, I want to exlude directory /home/waltdnes/.cache/ and
> >> all *.xz files in directory /home/waltdnes/pm/ The "--exclude="
> >> never worked. After much hair pulling, I was ready to give up on
> >> the exclude, and simply transfer all the unnecessary garbage.
> >>
> >> Then "I asked Mr. Google". It seems that I wasn't the only
> >> person running into problems. After some searching, I finally
> >> found a syntax that works...
> >>
> >> ####################################################
> >> #!/bin/bash
> >> export GZIP=-9
> >> tar cvzf wd.tgz --exclude ".cache/*" --exclude "pm/*.xz" waltdnes
> >> ####################################################
> >>
> >> Notes...
> >>
> >> 1) This is obviously not in line with the man page. Specifically,
> >> "--exclude" is followed by one space, not an equals sign.
> >>
> >> 2) ***THERE MUST BE EXACTLY ONE SPACE BETWEEN EACH WORD***
> >>
> >> 3) All directories and/or files to exclude must be listed as
> >> relative paths to the directory being tarred, i.e. last parameter
> >> on the command line.
> >>
> >> 4) I don't know the maximum line-length, which would limit the
> >> number of --exclude entries. In those cases, I wonder if
> >> "--exclude-from=FILE" works as "--exclude-from FILE".
> >>
> >
> > This works fine here with “tar (GNU tar) 1.34”:
> >
> > $ mkdir -p a/b
> > $ touch a/file a/b/file
> > $ touch a/file.xz a/b/file.xz
> > $ tree a
> > a
> > ??? b
> > ? ??? file
> > ? ??? file.xz
> > ??? file
> > ??? file.xz
> >
> > 1 directory, 4 files
> > $ tar -cvzf test.tar.gz --exclude="a/file" --exclude="a/b/*.xz" a
> > a/
> > a/file.xz
> > a/b/
> > a/b/file
> > $ tar -tf test.tar.gz
> > a/
> > a/file.xz
> > a/b/
> > a/b/file
> >
> > You can find out the maximum length of the command-line with
> > `getconf ARG_MAX`.
>
>
> But does it work with a space instead of = as well? According to the
> online manual page, it should work both ways.
>

Yes. It also works with 10 spaces.

--
Get my PGP key with `gpg --locate-keys tastytea@tastytea.de` or at
<https://tastytea.de/tastytea.asc>.

Re: [OT] tar exclude syntax tip [ In reply to ]

gtaylor at gentoo

May 5, 2021, 8:20 AM

Post #5 of 8 (536 views)

Permalink

On 5/5/21 7:33 AM, Walter Dnes wrote:
> 3) All directories and/or files to exclude must be listed as relative
> paths to the directory being tarred, i.e. last parameter on the
> command line.

This might not be very clearly articulated in the manual et al., but
once you are aware of it, you see evidence of it in multiple places.

I'm used to tar stripping the leading character if it's a forward slash
from paths. Thus tar actually works with relative paths based on the
(effective) working directory (which can be changed via the '-C' /
'--directory=' option).

So, after many years and even more battle scars, /I/ would /expect/ that
the path must be relative to the directory that tar is working on.
Perhaps I've simply survived more duels with tar. This is nothing
against you or anyone. After all, xkcd has this to say about tar: I'M
SO SORRY.

Link - TAR
- https://xkcd.com/1168/

--
Grant. . . .
unix || die

Re: [OT] tar exclude syntax tip [ In reply to ]

waltdnes at waltdnes

May 5, 2021, 8:28 AM

Post #6 of 8 (536 views)

Permalink

On Wed, May 05, 2021 at 04:03:03PM +0200, tastytea wrote
>
> This works fine here with ???tar (GNU tar) 1.34???:
>
> $ mkdir -p a/b
> $ touch a/file a/b/file
> $ touch a/file.xz a/b/file.xz
> $ tree a
> a
> ????????? b
> ????? ????????? file
> ????? ????????? file.xz
> ????????? file
> ????????? file.xz
>
> 1 directory, 4 files
> $ tar -cvzf test.tar.gz --exclude="a/file" --exclude="a/b/*.xz" a

That's files. What happens with directories, e.g.
tar -cvzf test.tar.gz --exclude="a/b/" a

I know I followed the manual. Actually, my problem seems to be with
directories. See
https://serverfault.com/questions/742514/running-the-tar-command-with-the-exclude-functionality-does-not-work-for-direc
where someone finds that...

tar hczf t.tar.gz * --exclude="./test1"

...doesn't work, but...

tar --exclude="./test1" -hczf t.tar.gz *

...does work!?!?!?

--
Walter Dnes <waltdnes@waltdnes.org>
I don't run "desktop environments"; I run useful applications

Re: [OT] tar exclude syntax tip [ In reply to ]

gentoo at tastytea

May 5, 2021, 9:00 AM

Post #7 of 8 (536 views)

Permalink

On 2021-05-05 11:28-0400 "Walter Dnes" <waltdnes@waltdnes.org> wrote:

> On Wed, May 05, 2021 at 04:03:03PM +0200, tastytea wrote
> >
> > This works fine here with ???tar (GNU tar) 1.34???:
> >
> > $ mkdir -p a/b
> > $ touch a/file a/b/file
> > $ touch a/file.xz a/b/file.xz
> > $ tree a
> > a
> > ????????? b
> > ??? ????????? file
> > ??? ????????? file.xz
> > ????????? file
> > ????????? file.xz
> >
> > 1 directory, 4 files
> > $ tar -cvzf test.tar.gz --exclude="a/file" --exclude="a/b/*.xz" a
>
> That's files. What happens with directories, e.g.
> tar -cvzf test.tar.gz --exclude="a/b/" a

With the trailing slash it doesn't work, but
`tar -cvzf test.tar.gz --exclude="a/b" a` works.

> I know I followed the manual. Actually, my problem seems to be with
> directories. See
> https://serverfault.com/questions/742514/running-the-tar-command-with-the-exclude-functionality-does-not-work-for-direc
> where someone finds that...
>
> tar hczf t.tar.gz * --exclude="./test1"
>
> ...doesn't work, but...
>
> tar --exclude="./test1" -hczf t.tar.gz *
>
> ...does work!?!?!?
>

That's because the files come last, the poster of the question has put
--exclude after the files.

When I'm in a, --exclude="./b" doesn't work, but --exclude="b" works.
tar doesn't seem to understand “./”.

--
Get my PGP key with `gpg --locate-keys tastytea@tastytea.de` or at
<https://tastytea.de/tastytea.asc>.

Re: [OT] tar exclude syntax tip [ In reply to ]

Warp_7 at gmx

May 5, 2021, 5:07 PM

Post #8 of 8 (536 views)

Permalink

Am Wed, May 05, 2021 at 09:33:58AM -0400 schrieb Walter Dnes:

> I'm passing on this solution to help others avoid my frustration and
> wasted time. If you've done "RTFM" on tar, you'll find out that "TFM"
> is broken or out-of-date or whatever, re: "--exclude=PATTERN". I'm
> fighting the urge to turn this into a rant.

Always reminds me of the good ol’ tar bomb:
https://xkcd.com/1168/

--
Gruß | Greetings | Qapla’
Please do not share anything from, with or about me on any social network.

What happens when you poop in an elevator?
It takes the shit to another leven.

Mailing List Archive

Mailing List Archive

Attached Files: