Robocopy — Problems with Junction Points

A junction point is a pointer to another location which looks like a directory. For example, you may create a junction point called C:\Data01 which points to C:\Apps\Database\Data01. Any access of the former will automatically reference the latter. Junction points can be very useful, but also dangerous and non-intuitive and should be used sparingly. This can present some issues with the powerful copy tool, Robocopy.

Destination Contains Large Number of Very Long Pathnames After Using Robocopy

If you use the Robocopy command in Windows Vista and above, the target drive may contain a large number of very long paths which cannot be deleted. The Robocopy command will also create many more files than exist on the source drive.

This occurs because Windows Vista changes many of the standard profile folders (eg. from C:\Documents and Settings to C:\Users) and to maintain compatibility with older applications which may have these paths hard-coded into them (as opposed to reading the correct folder from the registry), a number of junction points are added to the file system.

The particular problem that creates very long pathnames is that Windows Vista creates a recursive folder in each user profile by inserting a junction point which refers to its parent folder, and so any browsing through this junction point will continue forever as it simply keeps pointing back to itself.

When you use Robocopy to copy files, Robocopy traverses the junction points and, instead of recreating the junction point on the target, makes physical copies of all the files under the different path, resulting in many copies of the same file in different locations. When Robocopy traverses the recursive junction point, it keeps going until the path name exceeds the maximum allowed by the file system. However, the maximum allowed by the file system is much greater than that allowed by the shell (Windows Explorer) or the command processor (CMD.EXE) and so the files cannot be deleted.

Robocopy provides an option which prevents traversing junction points — /XJ — and it should be considered mandatory when using Robocopy with Windows Vista or later.

To remove very long paths, it is necessary to manually rename the offending folder names to the shortest available. For example, if you have a path that contains many “\Application Data\Application Data\Application Data\Application Data\Application Data\Application Data\Application Data” etc, rename each “Application Data” folder to “a” and when finished the paths (“\a\a\a\a\a\a\a”) will be short enough to delete via Windows Explorer. Another approach is to rename the top-most “Application Data” folder to some gibberish (eg skjdhsakhda), and on the next run Robocopy will remove it.

Robocopy Removes Files From Unrelated Location

Robocopy with the /MIR option makes a replica/mirror of the source at the destination. This means that:

• If a file is removed from the source, it will be removed from the destination;

• If a file is newer in the source, it will overwrite the older version of the file in the destination (to be expected); and

• If a file is older in the source, it will overwrite the newer version of the file in the destination (perhaps counter-intuitive).

An additional issue is that if the destination contains a junction point, Robocopy will traverse the junction and remove or replace those files.

Suppose the source folder is C:\Data and the destination is E:\Backup. Suppose also that there is a junction at E:\Backup\Documents which references E:\Documents.

If using Robocopy C:\Data E:\Backup /MIR, because the folder Documents doesn't exist under C:\Data, Robocopy will remove the folder from the destination after traversing it and removing all the files. This has the effect of deleting everything in E:\Documents.

A simple remedy is to check for junctions before allowing Robocopy to run:

dir E:\Backup /s | Findstr /I "<JUNCTION> <SYMLINK>" || Robocopy C:\Data E:\Backup /MIR /XJ

How this works: The directory listing of the destination is piped to Findstr and if no instances of <JUNCTION> or <SYMLINK> are found, the Robocopy command is allowed to run (ie the command after the double-pipe, ||).

Note that /XJ causes Robocopy to ignore junctions in the source, but apparently not in the destination.