Discussion:
Not all files in DFS Namespace replicating correctly
(too old to reply)
SuperEmpie
2006-11-03 11:14:43 UTC
Permalink
For a small customer of ours we implemented DFS Namespaces on 2 Windows
Server 2003 R2 Standard Edition servers. At first everything worked
fine, but now we get users reporting that when they sometimes create or
change files in one fysically seperate location, the data is not
replicated to the other site. The data needs to be at both locations.

When I create a file or folder on one of the locations, it is quickly
replicated to the other site. I have only tested this with small files.
If I understood the documentation on the Microsft website right, files
in use are skipped and staged/replicated at a later moment.
Replication is on 24/7 and working hours are from 8.00-17.00 hours. At
working time, the bandwidth used is less than when no one is working.

The first server was installed by someone else, the second server and
all necessary changes are done by myself.
The file and folder structure is a mess (too long paths exceding 260
characters etc.), but working on that. As far as I could tell, problems
also occured with paths smaller than 260 characters.

Some information regarding there infrastructure:

--start--
2 Windows Server 2003 R2 Standard Edition servers

Server 1 (HP Proliant ML350, dual CPU, 2GB mem, RAID1 for OS, RAID5 for
data) at location 1:
Domain Controller with all roles on it
DNS, DHCP, MS Exchage 2003, Terminal Services, License server, VPN
server
File and print server, Backup is also arranged here.

DFS Namespaces with 3 namespaces, AD is adprepped, configured with
Domain Based Namespace.
All DFS Hotfixes installed as specified by Microsoft.
Outlook 2003 is installed for Terminal Server use along with another
software package. Outlook is updated.

Server 2 (HP Proliant ML350, single CPU, 1GB mem, RAID1 for OS, RAID5
for data) at location 2:
Domain Controller
DNS, DHCP
No further roles on it

DFS Namespaces with 3 namespaces, AD is adprepped, configured with
Domain Based Namespace.
All DFS Hotfixes installed as specified by Microsoft.

Clients:
Clients (Windows XP Professional SP2) all have the failback hotfix
installed.

Network:
AD Sites are used. The 2 sites connect via a continuous 1MBit SDSL VPN
connection initiated by the routers (same type) on both locations. The
VPN connection is working fine.

AV Protection:
The virusscanner installed is Symantec AntiVirus 10.1 Multi-Tier
protection. According to Symantec this AV is compatible with DFS.

Backup:
Symantec (Veritas) BackupExec 10d
--end--

Does anyone have a clue why this can happen? I really would like to
help this customer.

SuperEmpie
Robert A Post, Jr (MSFT)
2006-11-08 22:18:49 UTC
Permalink
What types of files are you seeing this with? Is DFSR encountering Sharing
Violations (check the eventlog)?

DFSR won't replicated files that someone has an open handle to until it is
released, some apps may misbehave and hold file handles when it is not
necessary. In that case the file wouldn't be replicated until the file is
"closed" in the app or until the app is closed.

Thanks,

Robert A Post, Jr [MSFT]
Distributed File Service Replication Test
This posting is provided "AS IS" with no warranties, and confers no rights.
SuperEmpie
2006-11-11 08:36:28 UTC
Permalink
Post by Robert A Post, Jr (MSFT)
What types of files are you seeing this with? Is DFSR encountering Sharing
Violations (check the eventlog)?
DFSR won't replicated files that someone has an open handle to until it is
released, some apps may misbehave and hold file handles when it is not
necessary. In that case the file wouldn't be replicated until the file is
"closed" in the app or until the app is closed.
Thanks,
Robert A Post, Jr [MSFT]
Distributed File Service Replication Test
This posting is provided "AS IS" with no warranties, and confers no rights.
Hi Robert,

Thanks for your reply.

It happens with all sorts of files, from PDF, Word to AutoCAD files.
The amount of different file types used aren't that big.
There are sharing violations in the event log, but after 17.00 hours
local time, all people stop working and files are freed as they close
down their pc's (which should be enough to free open files). I can see
this trend in the event log, where as of about 17.00 hour there are no
more sharing violations. Also, in the weekends there is no work done
and still I find files that are not replicated. I have a 24/7
replication schedule.

Regards
Robert A Post, Jr (MSFT)
2006-11-15 01:01:25 UTC
Permalink
Sharing violation events will only appear every 8 hours I think due to some
exponential backoff of eventing. This may be showing you trend info that
is not correct. Get the backlogs from server to server to see if DFSR
thinks that they need to be replicated.

Are you sure that people are closing the programs and shutting down their
machines? If they leave the file open all night and continue to work on it
in the morn than the sharing violation would not be freed.

Are you aware that DFSR doesn't replicate some files? Encrypted files,
files with the TEMP attribute...

If these don't help please open a support call with PSS so we can look at
individual files and debug logs to see what the problem is

Thanks,

Robert A Post, Jr [MSFT]
Distributed File Service Replication Test
This posting is provided "AS IS" with no warranties, and confers no rights.
SuperEmpie
2006-11-16 09:15:21 UTC
Permalink
Post by Robert A Post, Jr (MSFT)
Sharing violation events will only appear every 8 hours I think due to some
exponential backoff of eventing. This may be showing you trend info that
is not correct. Get the backlogs from server to server to see if DFSR
thinks that they need to be replicated.
The only events I see concerning sharing violations, are these:

--begin event--
Event Type: Warning
Event Source: DFSR
Event Category: None
Event ID: 4304
Date: 16-11-2006
Time: 10:02:28
User: N/A
Computer: SCOPEFPEXCH01
Description:
The DFS Replication service has been repeatedly prevented from
replicating a file due to consistent sharing violations encountered on
the file. The service failed to stage a file for replication due to a
sharing violation.

Additional Information:
<cut>

For more information, see Help and Support Center at
http://go.microsoft.com/fwlink/events.asp.
--end event--

And only during working hours (8 AM - 6 PM)
Post by Robert A Post, Jr (MSFT)
Are you sure that people are closing the programs and shutting down their
machines? If they leave the file open all night and continue to work on it
in the morn than the sharing violation would not be freed.
Yes, I am sure.
Post by Robert A Post, Jr (MSFT)
Are you aware that DFSR doesn't replicate some files? Encrypted files,
files with the TEMP attribute...
I'm aware of that and made the necessary adjustments to exclude only
real temp files that don't need to be replicaticated, like file with a
~ in front, the File Filter is set to: ~*, *.tmp
Post by Robert A Post, Jr (MSFT)
If these don't help please open a support call with PSS so we can look at
individual files and debug logs to see what the problem is
What is PSS if I may ask and does it cost money?
Post by Robert A Post, Jr (MSFT)
Thanks,
Robert A Post, Jr [MSFT]
Distributed File Service Replication Test
This posting is provided "AS IS" with no warranties, and confers no rights.
Thanks,
Robert A Post, Jr (MSFT)
2006-11-17 00:44:42 UTC
Permalink
Even if you change the default file filter, encrypted files, file with TEMP
attribute, some reparse points are still not replicated...

A file set with the TEMP attrib is not common, but some programs may set
the TEMP attrib using win32 apis. You can determine if any of your non
replicating files are temp by looking at this blog post.
(http://blogs.technet.com/filecab/archive/2006/05/10/427837.aspx)

Did you try to get the backlogs using dfsrdiag? If there are no backlogs
then DFSR thinks the replicated folder is up to date.

PSS is product support...
http://support.microsoft.com/oas/default.aspx?ln=en-us&x=14&y=9&prid=5826&gp
rid=36983

Thanks,

Robert A Post, Jr [MSFT]
Distributed File Service Replication Test
This posting is provided "AS IS" with no warranties, and confers no rights.
SuperEmpie
2006-11-27 11:45:07 UTC
Permalink
Hello Robert,
Post by Robert A Post, Jr (MSFT)
Even if you change the default file filter, encrypted files, file with TEMP
attribute, some reparse points are still not replicated...
A file set with the TEMP attrib is not common, but some programs may set
the TEMP attrib using win32 apis. You can determine if any of your non
replicating files are temp by looking at this blog post.
(http://blogs.technet.com/filecab/archive/2006/05/10/427837.aspx)
I checked this and as far as I can see, this might be the issue. I have
tried a limited time program called fileTweak made by Febooti
(www.febooti.com).

I checked a non-replicating file using fsutil usn readdata
<path+filename> (as stated on the blog-link you gave me), and checked
the "File Attributes"-line in the output. One file had the value of
0x100.
Somewhere on the Internet I found this table in a program listing:
#DEFINE FILE_ATTRIBUTE_READONLY 1
#DEFINE FILE_ATTRIBUTE_HIDDEN 2
#DEFINE FILE_ATTRIBUTE_SYSTEM 4
#DEFINE FILE_ATTRIBUTE_ARCHIVE 32
#DEFINE FILE_ATTRIBUTE_NORMAL 128
#DEFINE FILE_ATTRIBUTE_TEMPORARY 256

I presumed this had some extra information for me to understand the hex
value
0x100 I thought should be 0xFF + 0x01 (256 + 1, looking at the table
above).

After I changed the Temporary attribute using Febooti's fileTweak, the
file I checked replicated within 3 seconds. It concerned an Adobe
Acrobat file, probably generated by "print to PDF"-application (not
sure about the exact orogin, but that should be of no concern yet).

Is there any way I can control this using DFS or Windows? As you state
some programs may set this attribute, so I would like to determine the
temp-files on a file-extension basis only, if possible, in stead on a
file attribute basis.
Could this be considered a small short-coming in DFS Namespaces? I
can't imagine I'm the only one having these kind of problems.
Post by Robert A Post, Jr (MSFT)
Did you try to get the backlogs using dfsrdiag? If there are no backlogs
then DFSR thinks the replicated folder is up to date.
Backlogs seem to be ok. Had a message on one server stating there were
6 items in backlog. Onthe other server there were none.
Post by Robert A Post, Jr (MSFT)
PSS is product support...
http://support.microsoft.com/oas/default.aspx?ln=en-us&x=14&y=9&prid=5826&gp
rid=36983
Thanks for clearing that :-)
Post by Robert A Post, Jr (MSFT)
Thanks,
Robert A Post, Jr [MSFT]
Distributed File Service Replication Test
This posting is provided "AS IS" with no warranties, and confers no rights.
Thanks so far,
SuperEmpie
Ned Pyle [MSFT]
2007-01-23 18:16:10 UTC
Permalink
Are the files in question JPEG, JPG, XLT, X, or XLSX?
--
Ned Pyle
Microsoft Enterprise Platform Support
This posting is provided "AS IS" with no warranties, and confers no rights.
Please read http://www.microsoft.com/info/cpyright.htm for more information.
SuperEmpie-
Any luck getting this resolved. I'm having the same problem that you're
having. Random files are not being replicated between two servers. When
I
run the fsutil i get file attribute of 0x80 (I believe its read attribute)
The DFS Replication service detected that a file was changed on multiple
servers. A conflict resolution algorithm was used to determine the winning
file. The losing file was moved to the Conflict and Deleted folder.
Post by SuperEmpie
Hello Robert,
Post by Robert A Post, Jr (MSFT)
Even if you change the default file filter, encrypted files, file with TEMP
attribute, some reparse points are still not replicated...
A file set with the TEMP attrib is not common, but some programs may set
the TEMP attrib using win32 apis. You can determine if any of your non
replicating files are temp by looking at this blog post.
(http://blogs.technet.com/filecab/archive/2006/05/10/427837.aspx)
I checked this and as far as I can see, this might be the issue. I have
tried a limited time program called fileTweak made by Febooti
(www.febooti.com).
I checked a non-replicating file using fsutil usn readdata
<path+filename> (as stated on the blog-link you gave me), and checked
the "File Attributes"-line in the output. One file had the value of
0x100.
#DEFINE FILE_ATTRIBUTE_READONLY 1
#DEFINE FILE_ATTRIBUTE_HIDDEN 2
#DEFINE FILE_ATTRIBUTE_SYSTEM 4
#DEFINE FILE_ATTRIBUTE_ARCHIVE 32
#DEFINE FILE_ATTRIBUTE_NORMAL 128
#DEFINE FILE_ATTRIBUTE_TEMPORARY 256
I presumed this had some extra information for me to understand the hex
value
0x100 I thought should be 0xFF + 0x01 (256 + 1, looking at the table
above).
After I changed the Temporary attribute using Febooti's fileTweak, the
file I checked replicated within 3 seconds. It concerned an Adobe
Acrobat file, probably generated by "print to PDF"-application (not
sure about the exact orogin, but that should be of no concern yet).
Is there any way I can control this using DFS or Windows? As you state
some programs may set this attribute, so I would like to determine the
temp-files on a file-extension basis only, if possible, in stead on a
file attribute basis.
Could this be considered a small short-coming in DFS Namespaces? I
can't imagine I'm the only one having these kind of problems.
Post by Robert A Post, Jr (MSFT)
Did you try to get the backlogs using dfsrdiag? If there are no backlogs
then DFSR thinks the replicated folder is up to date.
Backlogs seem to be ok. Had a message on one server stating there were
6 items in backlog. Onthe other server there were none.
Post by Robert A Post, Jr (MSFT)
PSS is product support...
http://support.microsoft.com/oas/default.aspx?ln=en-us&x=14&y=9&prid=5826&gp
rid=36983
Thanks for clearing that :-)
Post by Robert A Post, Jr (MSFT)
Thanks,
Robert A Post, Jr [MSFT]
Distributed File Service Replication Test
This posting is provided "AS IS" with no warranties, and confers no rights.
Thanks so far,
SuperEmpie
Ned Pyle [MSFT]
2007-01-23 18:27:13 UTC
Permalink
Oops, also XLS (duh on me)
--
Ned Pyle
Microsoft Enterprise Platform Support
This posting is provided "AS IS" with no warranties, and confers no rights.
Please read http://www.microsoft.com/info/cpyright.htm for more information.
Post by Ned Pyle [MSFT]
Are the files in question JPEG, JPG, XLT, X, or XLSX?
--
Ned Pyle
Microsoft Enterprise Platform Support
This posting is provided "AS IS" with no warranties, and confers no
rights. Please read http://www.microsoft.com/info/cpyright.htm for more
information.
SuperEmpie-
Any luck getting this resolved. I'm having the same problem that you're
having. Random files are not being replicated between two servers. When
I
run the fsutil i get file attribute of 0x80 (I believe its read attribute)
The DFS Replication service detected that a file was changed on multiple
servers. A conflict resolution algorithm was used to determine the winning
file. The losing file was moved to the Conflict and Deleted folder.
Post by SuperEmpie
Hello Robert,
Post by Robert A Post, Jr (MSFT)
Even if you change the default file filter, encrypted files, file with TEMP
attribute, some reparse points are still not replicated...
A file set with the TEMP attrib is not common, but some programs may set
the TEMP attrib using win32 apis. You can determine if any of your non
replicating files are temp by looking at this blog post.
(http://blogs.technet.com/filecab/archive/2006/05/10/427837.aspx)
I checked this and as far as I can see, this might be the issue. I have
tried a limited time program called fileTweak made by Febooti
(www.febooti.com).
I checked a non-replicating file using fsutil usn readdata
<path+filename> (as stated on the blog-link you gave me), and checked
the "File Attributes"-line in the output. One file had the value of
0x100.
#DEFINE FILE_ATTRIBUTE_READONLY 1
#DEFINE FILE_ATTRIBUTE_HIDDEN 2
#DEFINE FILE_ATTRIBUTE_SYSTEM 4
#DEFINE FILE_ATTRIBUTE_ARCHIVE 32
#DEFINE FILE_ATTRIBUTE_NORMAL 128
#DEFINE FILE_ATTRIBUTE_TEMPORARY 256
I presumed this had some extra information for me to understand the hex
value
0x100 I thought should be 0xFF + 0x01 (256 + 1, looking at the table
above).
After I changed the Temporary attribute using Febooti's fileTweak, the
file I checked replicated within 3 seconds. It concerned an Adobe
Acrobat file, probably generated by "print to PDF"-application (not
sure about the exact orogin, but that should be of no concern yet).
Is there any way I can control this using DFS or Windows? As you state
some programs may set this attribute, so I would like to determine the
temp-files on a file-extension basis only, if possible, in stead on a
file attribute basis.
Could this be considered a small short-coming in DFS Namespaces? I
can't imagine I'm the only one having these kind of problems.
Post by Robert A Post, Jr (MSFT)
Did you try to get the backlogs using dfsrdiag? If there are no backlogs
then DFSR thinks the replicated folder is up to date.
Backlogs seem to be ok. Had a message on one server stating there were
6 items in backlog. Onthe other server there were none.
Post by Robert A Post, Jr (MSFT)
PSS is product support...
http://support.microsoft.com/oas/default.aspx?ln=en-us&x=14&y=9&prid=5826&gp
rid=36983
Thanks for clearing that :-)
Post by Robert A Post, Jr (MSFT)
Thanks,
Robert A Post, Jr [MSFT]
Distributed File Service Replication Test
This posting is provided "AS IS" with no warranties, and confers no rights.
Thanks so far,
SuperEmpie
Ned Pyle [MSFT]
2007-01-23 21:39:58 UTC
Permalink
Actually, it should. There was a specific hotfix for your issue (917622),
but it was rolled up into this same file (the DFSR.EXE core service
executable). So, you could have gotten an older file written just for this
problem, or gotten this one which will fix your issue, and many others.

Let me know if it doesn't though!
--
Ned Pyle
Microsoft Enterprise Platform Support
This posting is provided "AS IS" with no warranties, and confers no rights.
Please read http://www.microsoft.com/info/cpyright.htm for more information.
Ned,
The file has an xls extension. I've opened a ticket with PSS and they
provided me with a patch that they say should address the issue which I
don't
think it will. The KB article is 925377. I can email you my PSS incident
number if you like so that you can look it over.
Post by Ned Pyle [MSFT]
Are the files in question JPEG, JPG, XLT, X, or XLSX?
--
Ned Pyle
Microsoft Enterprise Platform Support
This posting is provided "AS IS" with no warranties, and confers no rights.
Please read http://www.microsoft.com/info/cpyright.htm for more information.
SuperEmpie-
Any luck getting this resolved. I'm having the same problem that you're
having. Random files are not being replicated between two servers.
When
I
run the fsutil i get file attribute of 0x80 (I believe its read attribute)
The DFS Replication service detected that a file was changed on multiple
servers. A conflict resolution algorithm was used to determine the winning
file. The losing file was moved to the Conflict and Deleted folder.
Post by SuperEmpie
Hello Robert,
Post by Robert A Post, Jr (MSFT)
Even if you change the default file filter, encrypted files, file
with
TEMP
attribute, some reparse points are still not replicated...
A file set with the TEMP attrib is not common, but some programs may set
the TEMP attrib using win32 apis. You can determine if any of your non
replicating files are temp by looking at this blog post.
(http://blogs.technet.com/filecab/archive/2006/05/10/427837.aspx)
I checked this and as far as I can see, this might be the issue. I have
tried a limited time program called fileTweak made by Febooti
(www.febooti.com).
I checked a non-replicating file using fsutil usn readdata
<path+filename> (as stated on the blog-link you gave me), and checked
the "File Attributes"-line in the output. One file had the value of
0x100.
#DEFINE FILE_ATTRIBUTE_READONLY 1
#DEFINE FILE_ATTRIBUTE_HIDDEN 2
#DEFINE FILE_ATTRIBUTE_SYSTEM 4
#DEFINE FILE_ATTRIBUTE_ARCHIVE 32
#DEFINE FILE_ATTRIBUTE_NORMAL 128
#DEFINE FILE_ATTRIBUTE_TEMPORARY 256
I presumed this had some extra information for me to understand the hex
value
0x100 I thought should be 0xFF + 0x01 (256 + 1, looking at the table
above).
After I changed the Temporary attribute using Febooti's fileTweak, the
file I checked replicated within 3 seconds. It concerned an Adobe
Acrobat file, probably generated by "print to PDF"-application (not
sure about the exact orogin, but that should be of no concern yet).
Is there any way I can control this using DFS or Windows? As you state
some programs may set this attribute, so I would like to determine the
temp-files on a file-extension basis only, if possible, in stead on a
file attribute basis.
Could this be considered a small short-coming in DFS Namespaces? I
can't imagine I'm the only one having these kind of problems.
Post by Robert A Post, Jr (MSFT)
Did you try to get the backlogs using dfsrdiag? If there are no backlogs
then DFSR thinks the replicated folder is up to date.
Backlogs seem to be ok. Had a message on one server stating there were
6 items in backlog. Onthe other server there were none.
Post by Robert A Post, Jr (MSFT)
PSS is product support...
http://support.microsoft.com/oas/default.aspx?ln=en-us&x=14&y=9&prid=5826&gp
rid=36983
Thanks for clearing that :-)
Post by Robert A Post, Jr (MSFT)
Thanks,
Robert A Post, Jr [MSFT]
Distributed File Service Replication Test
This posting is provided "AS IS" with no warranties, and confers no rights.
Thanks so far,
SuperEmpie
Continue reading on narkive:
Loading...