[Bug-tar] Bug? Where? Why? Why so many files changing as we read them?

[Linda A. Walsh]

This may not be a tar bug, but I am trying to figure out
what is going on and/or why so many 'changes'...

using tar V1.28 on linux-4.1.0

Reading from a Win7-SP1 workstation via 'CIFS' (mounted fs)

tar cf ~/Appdata.tar --acls --xattrs Appdata

In my "Windows Home directory" as seen as a mounted file system via CIFS.

//Athenae/C/ /athenae/ cifs
user,noauto,rw,uid=x,gid=x,nocase,serverino,creds=xxx,setuids 0 0

tar cf ~/Appdata.tar --acls --xattrs Appdata

And got **78** lines (out of 64389 files in 12357 directories)
that claim "something changed".
Note: I wasn't running any of the below related apps as tar ran:

....
tar: Appdata/Local/Google/Chrome/User Data/Default/Cache: file changed
as we read it
tar: Appdata/Local/Google/Chrome/User Data/Default: file changed as we
read it
tar: Appdata/Local/Google/Chrome/User Data: file changed as we read it
tar: Appdata/Local/Microsoft/ehome/Art Cache: file changed as we read it
tar: Appdata/Local/Microsoft/Media Player/Art Cache/LocalMLS: file
changed as we read it
tar: Appdata/Local/Microsoft/OneNote/12.0/Backup/Personal Notebook: file
changed as we read it
tar: Appdata/Local/Microsoft/OneNote/12.0/Backup: file changed as we read it
...
tar: Exiting with failure status due to previous errors
----


Of minor note, "find" commands run locally via cygwin
and on linux (via CIFS) both show the same # files+dirs
(76746) -- showing a 1:1 mapping.

I did an "ls -l" from both sides (cyg v. linux+CIFS) that shows
the last mod times -- Looked at
"Appdata/Local/Microsoft/OneNote/12.0/Backup"
and the subdir "Personal Notebook":

Oct 15 2012 Backup
Mar 18 2015 Personal Notebook

I.e. neither looks like they changed recently.

So why the messages?



Thanks!

[Pavel Raiskup]

Post by Linda A. Walsh
This may not be a tar bug, but I am trying to figure out
what is going on and/or why so many 'changes'...

[..]

Post by Linda A. Walsh

tar cf ~/Appdata.tar --acls --xattrs Appdata

And got **78** lines (out of 64389 files in 12357 directories)
that claim "something changed".
....
tar: Appdata/Local/Google/Chrome/User Data/Default/Cache: file changed
as we read it

Try 'stat' on this file before/after running 'tar -c'. See '10.1.3 Race
conditions' section in documentation [1]. Either make sure tar is the
only process working with those files (globally as that is CIFS mount
point, not only on your machine) or pass there
'--warning=no-file-changed'.

[1] https://www.gnu.org/software/tar/manual/tar.html

Pavel

[Linda A. Walsh]

Try 'stat' on this file before/after running 'tar -c'. See '10.1.3 Race
conditions' section in documentation [1].

----
Well the 78 messages come out sprinkled throughout a 60-70K file dump.
It doesn't seem a 'stat' would be practical to try to isolate 1 in 1000
files for a change that "might" be occurring...

I'm already familiar with the concepts in section 10.1.3, but as
I mentioned, those files belonged to apps that weren't in use, and
in come cases haven't been used for quite some time. Their
last-mod times indicated that all the files
and the directory hadn't been changed in some time.

FWIW, the volume it was accessing has 'last-accessed' time turned off,
but I think it also has semantics similar to linux kernel's default
last-access time update (i.e. if writing to inode and updating other stuff,
then make sure last-access time is no earlier than other time/date fields
in the inode. The only think accessing them would have been tar, and the
occasional cosmic ray, but tar should handle the 1st case... ;-)

Either make sure tar is the
only process working with those files (globally as that is CIFS mount
point, not only on your machine)

----
That's the thing. It *is* only a CIFS mount point on my machine.
I'm the only user on either of the machines. The link between the machines
is a cross-over wire (no routers or switches) @ 10Gb, so the connection is
the Win7 workstation's is a dedicated link to a linux server, who's only
function is to handle the Win7's internet & disk needs.... so -- single
user.

I.e. "--warning=no-file-changed" would only hide the symptom -- not
explain
or prevent the cause. ;-(

[Pavel Raiskup]

Post by Linda A. Walsh

Try 'stat' on this file before/after running 'tar -c'. See '10.1.3 Race
conditions' section in documentation [1].

----
Well the 78 messages come out sprinkled throughout a 60-70K file dump.
It doesn't seem a 'stat' would be practical to try to isolate 1 in 1000
files for a change that "might" be occurring...

One file is enough to prove that the "at least one file" has really
changed and tar is most likely not guilty, src/create.c source:

if ((timespec_cmp (get_stat_ctime (&final_stat), original_ctime) != 0
/* Original ctime will change if the file is a directory and
--remove-files is given */
&& !(remove_files_option && is_dir))
|| original_size < final_stat.st_size)
{
WARNOPT (WARN_FILE_CHANGED,
(0, 0, _("%s: file changed as we read it"),
quotearg_colon (p)));
set_exit_status (TAREXIT_DIFFERS);
}

So in case the "guilty" file has not changed its 'ctime' or its size, tar
should not complain. If there is a bug in GNU tar, I would suggest you to
use debugger and walk precisely through the cited code.

[Paul Eggert]

One possibility is that the file system is buggy, and that reading a
file (and therefore modifying st_atime) also modifies st_ctime as a side
effect. Often this sort of thing is delayed, that is, you read a file
and its st_atime is updated immediately, but its st_ctime is updated
after a delay of a few seconds. This would explain the observed behavior.

See, for example:

https://bugzilla.redhat.com/show_bug.cgi?id=1058526

which does indeed illustrate remote file-system bugs that can cause the
behavior in question. (Mark O'Keefe's proposed change to tar isn't
right; there are security reasons to worry about ctime as opposed to mtime).

[Linda Walsh]

Post by Paul Eggert
One possibility is that the file system is buggy, and that reading a
file (and therefore modifying st_atime) also modifies st_ctime as a
side effect. Often this sort of thing is delayed, that is, you read a
file and its st_atime is updated immediately, but its st_ctime is
updated after a delay of a few seconds. This would explain the
observed behavior.

----
AFAIK (but with MS, who really knows anything?), MS adopted a similar
system to the linux
kernel in regards to updating last-access times when atime updating is
turned off. I.e.
I thought that the kernel opportunistically updates last-access if
something else changes in
the inode to be no earlier than the ctime or modtime fields.

I just read last week in looking at MS notes on NTFS implementation,
that it does
something similar when don't have last-access turned off -- but it may hold
off on the writing out of the last access time for up to an hour --
trying to wait
for a time when one of the other times is updated.

But that said -- I don't get the same messages when I run tar on the
client-MS
machine. Instead I get failures to read ACL's and ext-attrs... which is
maybe worse.
But it's only when I use the linux CIFS to access the remote NTFS that
tar gets upset.
So I don't think it is the source FS.

Post by Paul Eggert
https://bugzilla.redhat.com/show_bug.cgi?id=1058526
which does indeed illustrate remote file-system bugs that can cause
the behavior in question. (Mark O'Keefe's proposed change to tar isn't
right; there are security reasons to worry about ctime as opposed to mtime).

Well -- I have yet to try to use stat on all the files b4 and aftr ...
keep getting sidetracked.