Using a .NET FileWatcher with a dfs properly

There is not a lot of information with practical experience about the Microsoft .NET File Watcher Technology available in the internet. So here some advices.

How it works

It is basically a wrapper around the windows api function ReadDirectoryChangesW.

Possible restrictions with remote locations

There might be several limitations in place that you usually cannot see:

  • Latency raises the chance for a buffer overflow.
  • The buffer size might be artifically restricted to a size smaller than 64 kB.
  • The implementation might miss the ReadDirectoryChangesW function or it might be implemented incorrectly (for example if you are dealing with a linux/unix server)
  • The server might shut down incorrectly. In such a case the FileWatcher might be unaware of the fact that his handle is invalid and just stop reporting changes from that point on. Unfortunately it will usually also not raise any error and there is no timeout. The only solution I know about is a frequent reregistration (setting EnableRaising events to false and the to true). But if you do this, you need to pay attention not to loose any events.

My advice would be to do some testing whenever you are using the FileWatcher with a remote location. You could create a programm that generates some files and verify that the events are properly raised

Buffer

The FileWatcher uses an internal buffer of 8 kB. It is possible to raise it to 64 kB. This should be done if you are monitoring an entire disk where there can be many events. However it is costly because it will be nonpaged memory. When you get the event it frees some space in the buffer. This means it is really important to process the events as fast as possible. Don’t do any check in the event handler. Add the event to another queue of your own and process it there asynchronously. If the buffer overflows you get an error and you

The size of the elements of the buffer is 12 bytes + two times the length of the path. In the worst case it can hold only 15 events (all path 260 characters) when configured with a buffer size of 8 kB.

Error Handling

You need to register to the Error event.

One reason why an error could be raised is that the buffer has been filled up. In that case you will get an InternalBufferOverflowException. It is likely that you missed events.

You might also get an error when a remote location shuts down or some network error happens. In such a case it is usually advisable to reregister the watcher. Otherwise it might be the case that it doesn’t raise any more events (invalid handle).

Locking

You will receive many events when the file it concerns is still locked. Therefore you need some logic to wait until the file is “free” before you do whatever you want to do.

Configuring it correctly

You should set the NotifyFilter property to as few changes as possible in order not to stress the buffer to much. You can also create multiple instances of the FileWatcher each monitoring the same directory configured for different change types. In that case the logic further down the stream will be more complex to merge all those events.

If you only need to watch some filetypes you can also set the filter property

Furthermore the property IncludeSubdirectories should be set if you want to monitor an entire drive or folder structure

Access Rights special case

Another strange aspect of the FileWatcher is that it won’t respect the access rights of the user registering the FileWatcher. Thus you can get events for files that your program is not even allowed to see. You should filter such events or at least be prepared to see them.

Using it with a Windows DFS based on Windows Server 2012 R2

I used it in a configuration which had multiple data servers behind a common root. It worked without any problem, except for the buffer overflows if lots of activity was going on.

I got events about DFSRPrivate folders. This is a typical case of the missing consideration of access rights. The application was not allowed to access those “internal” files and needed to skip them

Some events happened with temporary office files (when the users opens word- or excel-documents). I also filtered them out.

More information

  • http://blogs.msdn.com/b/winsdk/archive/2015/05/19/filesystemwatcher-follies.aspx
  • https://msdn.microsoft.com/en-us/library/system.io.filesystemwatcher%28v=vs.110%29.aspx
  • http://stackoverflow.com/questions/13916595/is-it-really-that-expensive-to-increase-filesystemwatcher-internalbuffersize

Run .NET Applications in 32-Bit Mode under 64-Bit Windows

Sometimes it is necessary to run .NET applications in 32-Bit mode. One example is the usage of Adobe Reader Interop Libraries which will crash if they run in 64-Bit mode.

This will force all .NET applications to be run in 32-Bit mode.