Cannot overwrite files when free disk space is insufficient

I cannot overwrite old files/folders, particularly on external devices, when the remaining free disk space does not meet the amount I am copying even if I know there is enough room when replacing all of the old files.

I like to be able to overwrite an entire folder because I have several sub-folders which I often make changes to on my machine and can't always keep track of what I am changing in each of these folders.

An example is if I have a 4GB flash drive, I have a folder that amounts to 3GB, I add about 10MB worth of files to it in random places, and just want to overwrite the older folder on the flash drive which would then have 1GB free space left.

Windows complains and does not just let me choose to overwrite the folder to combine the newer stuff in it and do not wish to delete the folder on the flash drive in case something in the other folder went missing.

How can I turn this feature off? Even a suggested registry entry will be fine with me.

 

That is correct. You can not simply 'over write'. You must have enough space to write the file to. Storage is cheap. Spend the ten bucks and get a second thumb drive.

 

No, you misunderstand. It's that there is plenty of space and I am just trying to overwrite files in the same directory rather than have to delete and then paste again!

And the flash drive story was an example. My problem applies to my external hard drives (250GB WD Passport & 320GB WD MyBook) when copying far larger amounts of data. I have 8GB in a folder with MANY subfolders and I keep a duplicate on my local hard drive and update my files in it as necessary. Then I will add about 1 GB worth more of data to that folder in random directories and I'll want to back that up. My hard drive will have 6GB free of space which means after I copy and overwrite everything, it will drop down to 5GB free space.

If I try this in Windows Vista or Windows 7, it will see that my new folder is 9GB large and tell me I cannot copy it anyway. When I used Windows XP this was NEVER a problem! This is actually quite annoying and frustrating!

 

You need to delete them first, otherwise the space isn't available. You might be able to use some kind of synchronizing software and do it file by file, but, if you want to do it all at once, the two choices are 'Copy' and 'Move'. Both require that space is available on the target disk.

 

Well, see, that's the problem. Previous versions of windows never made me delete the files before copying them again to simply replace them. This is the purpose to copy them without having to do them file by file.

 

Have you tried opening the folder and selecting all the files with Ctrl-A and the copy or move?

 

Hmmm... you might be on to something there. I might give that a try. I'll let you know how it works later when I come across it again. Thanks for the idea. If I can get it to work that way it won't be such a big deal.

 

Nope. No cigar. I tried pasting 513MB worth of videos to a 1 GB flash drive I had laying around to make a small copy and then do the same copy again to overwrite the exact same video files I just copied and it still gave me that screen. Arrrrgghh! Tell me there is a fix for this!

 

No version of Windows has EVER allowed you to physically overwrite a file. They all perform a copy and subsequent delete of the old file. Any other method would be much to risky of potential data loss.

Gary

 

That's exactly right. I know that's exactly how it works, but earlier OSes didn't check for open space before assuming you are just replacing already existing files and have the OS do all that work for you.

I would just like to be able to set it so that the OS only complains about lack of space when the drive has actually hit it's limit like all the other OSes used to.

I have actually tried the old method via the command prompt and used the old DOS command 'xcopy /e c:\xxx f:\xxx' and that worked okay, but for some reason on one file I got an access denied error. I have no idea why I got that error, but I'm wondering if it was just something in my folder structure or a protected file that caused that one particular file at that particular time not to copy.

 

If you understood that, why does this surprise you:

There was no room to make the initial copy that would be the first step in the process. The first copy used over half the flash drive. Without deleting the original file there is no room to do the copy.

Gary

 

No, there is room to do the copy because it is a simple replace. If I boot to windows 95, NT4.0, 98, 2000, ME or XP and do the same exact thing, the file will allow to be replaced without any hassle. I will be able to copy directly over the old with the new.

This means, with Windows Vista or 7, I will have to always have double the space open to make a copy of a folder with MANY subfolders and MANY files in each folder to make the replace to COMBINE small added files into that huge file collection. Meaning If I have a folder with lots of files in many sub-folders equaling up to 200GB, I will need a 500GB hard drive and always leave at least 200GB free in order to replace the folder with another on my other PC's with different added files in different locations. To make this clearer, after copying those folders in that manner from both PC's to the external to add those minor, yet several changes/additions (say, under 1GB worth of files), I will still have 299GB free on the drive after the copy and replace.

This was NEVER an issue with ANY of the previous OSes.

Look! Try it yourself between Vista/7 against the other mentioned OSes and you will see what I mean and why this is so frustrating!

 

might sound silly, but do you have disk quota turned on? You can check by right clicking your hard drive in my computer and selecting properties.

 

No, disk quota is not checked. It is off.

Would it help to turn it on, perhaps?


edit: That was actually a silly question on my part because I know disk quota is used to limit the amount one can use. heh, hard tap on me!

 

I think you are wrong in your assessment of how the old OS worked, regarding the 513MB file and the 1 GB stick. I have been synchronizing the "My Documents" folder between a laptop and desktop machine almost daily for several years using SureSync. The single largest file has always been my Outlook.PST, several hundred MB. Under Win2k and WinXP, at times I have had insufficient space to replace that file on the target machine UNLESS I deleted the target file manually first. As soon as I deleted the target file I could do the copy. For example if the OUTLOOK.PST file was 235MB, I had to have at least 235MB free on the target drive even if the old file on the target drive was 234MB. This is your "simple replace" scenario.

Now to the issue with folders, in folders, I need to look into that a bit. Are you saying that Vista and Win7 require empty space the equivalent size of the ENTIRE folder structure? Or empty space equivalent to the size of the largest SINGLE file in the process?

As an aside, why are you copying the entire folder structure each time? Why not use something like SyncToy (free from Microsoft)? It will copy ONLY the changed files.

Gary

 

I don't want to tell you that you're wrong or anything, but that has never happened to me where I had to delete before copying again with those OSes. I believe you are having that trouble simply due to folder sync. I have never had that issue and I have also never used folder sync myself for anything. I'm just saying here that folder sync was where you were getting that error and only with sync.

That case may be true for synchronization, but never had that happen just doing simple overwriting.

 

You raise a good point! It could very well be the tool I am using. I'll see if I can test it out this week. But I didn't think the OS would blow away the old file before copying in the new one. Just seems REALLY dangerous.

My side question still stands though, why are you copying the entire folder structure each time?

Gary

 

Because the actual operation is something like follows:
1)Copy in the new files onto unused space on the harddisk
2)Change file pointer to new files
3)Invalidate old file location and mark it as unused

So if 1) fails 2) & 3) cannot proceed.

 

I know it may sound unusual, but the way I am doing this is actually easier the way I back up my stuff.

For instance, I have a laptop and a desktop where I have all of my utilities and programs handy where I never have to pull out a CD to install anything.

This is an example of my folder structure

WINTOOL\
UTIL
NAV2009
ATI


There are way more folders than this, but this is how it might happen.
I find online a new utility (using laptop) for converting videos and I say "OOOHHH! Neat!" and I download it, try it, like it, and decide to keep it. So... I copy it to my UTIL folder. I will find several other progs like this and wish to store them there.

Then I go to my desktop computer with the same wintool folder and structure and download an updated video driver for my ati card. I will then put that in the ATi folder, of course .

This will happen on and off and I will not be able to keep track of all of those additions in, say, a month of time before backing it up.

So I will connect my external HDD to my laptop and copy the WINTOOL folder with the added programs to the one on the external. Windows XP and prior would ask me to overwrite and I would say 'Yes to all' and just wait out the process, but the stuff on my desktop has not been added to this compilation.

So I will then connect the external HDD to my desktop and repeat to add the files from there that were not on my laptop and were obviously not on the external yet. Wallah! Completed backup on external so everything is in one convenient place and I put my external away, say, in my closet or somewhere... you get the idea.

Well, between the downloaded programs you would have:

ATi driver - 30 MB
video converter - 15 MB

Together that would equal 45 MB total added to the WINTOOL folder compilation.

My WINTOOL folder is currently 8GB large.
My external HDD has 6 GB free.

I think that 6 GB well covers the extra 45MB!!!!!

That's why this is so aggravating!!!

Something so simple has become complicated!

 

That's precisely what I have been saying has happened in the OS dating back to Win98. But Slayer366 keeps insisting that it is not the case and the old file is deleted first. I just cannot, for the life of me, fathom that the OS would do it his way. It makes no sense from a safety standpoint. But I have not had the chance to verify it.

Gary

Slayer366, even given the structure and nature of your copies I still think you'd be much better off using SyncToy. Have you looked at it at all?

Gary

 

Trust me, my good friend, it works. I have never, but NEVER had any file corruptions doing this. I have used this method for many years starting with (strap yourselves in) DOS 5.0 as my first OS on my Tandy 1000 RSX 386@25Mhz.

'Overwrite files (Yes, No, yes to All)?' A

worked every time UNTIL the destination actually ran out of space only upon the next file to be copied where there, then, was not enough space for that following file. BEAUTIFUL system! What is this now suddenly?

To be honest, no. I always have done it manually and either perhaps out of nostalgia or just old method, I have never liked to rely on another program or the OS itself to do a task that I know I can do myself. I just don't feel comfortable using a system like sync because with copying files I know I can keep a closer eye on what gets copied.

Don't get me wrong, I could be mistaken about the sync thing, but I am just too afraid to switch to it, especially when I have been using this old method that has worked just fine for me for the longest time.

 

Here's a video I prepared for you all to watch. Maybe this will help everyone understand better where I am coming from.

http://www.youtube.com/watch?v=xFqyDBZMwjs

I even showed my friends this Windows 7 change and they are not happy with what they see either. One friend I showed this to said he will not switch from XP now because he also backs his stuff up the same way I do. Go figure...

I just hope that there is a registry entry or something so I can change this.

I've even tried 3rd party browser apps and they did nothing as far as bypassing this.

 

This is the safe way for file overwriting
1)The new folder/file may not have the same disk space allocated so a direct overwrite can result in insufficient allocation space or fragmentation as the files are spread across multiple sectors.
2)If the old files are deleted first if the copy fails it will be disastrous

2 alternatives
1)Manually delete the old files first
2)Select ignore/skip when there is a old copy assume only new files get added while existing ones are not updated.

An improvement to the algorithm is do the 1),2) and 3) for small branches each time when a big tree of folders is copied so you won't run out of space quickly because old spaces are freed up each time.
However this can result in read write speed penalty as each time read/writes are queued, penalty is added each time instead of 1 NCQed read write.

 

That is the case when there is not enough space. So I will then find things I do not any longer need (Such as large isos I've compiled in the past) and delete them and start the copy over. Then it will be the same as if there was not an error in the first place. Also because only existing files in the folder being copied will only affect the existing files in the destination folder. The files in the destination folder that are not in the source folder will not be touched.

Two things here. If I understand what you mean about the allocated space, that is determined directly on the file system itself and will adjust accordingly to the file system on the drive. Allocations get applied in real time, the tables do not copied from another file system.

fragmentation and spreading across multiple sectors - true, very true, but it's not like you cannot run a defrag and c'mon, you can't say that doesn't happen regularly even when copying clean. Besides, since Windows XP these OSes were very good at making sure your files stay written well contiguous to your drives.

If the fragmentation thing was dangerous, then DOS, Windows 95 & 98 users were in big trouble!

I'm not exactly sure what you mean by this unless you are talking about the actual folder tree reserving space and making an allocation on the FS. Even if that is true;

1) Do you know how little space that may take up each time? Maybe bytes!
2) You can run a chkdsk if worst comes to worst.

I still don't know, however, what you mean by running out of space quickly.
You lost me there completely. I honestly have no idea what you're talking about. An 8GB folder is 8GB no matter how you want to look at it. Sorry... I REALLY don't know what you're talking about.

and chance loosing important documents that were not saved in the source folder that I will be copying! I won't be thinking that at the time, delete the folder with those docs, and come back to my folder later to find it and my critical documents won't be there, but the newly added stuff will be only what's there. Good idea!

 

I watched you video and you are wrong about synchronization. It is not automatic, you can control when it synchs the files. And you can specify what folder is the master and which is the target. I am not sure why you are so reluctant to use a tool that is designed to do the job you are attempting. The reuslts and inherent dangers are absolutely no different than what you are doing now.

Yes it appears the behavior of Win7 is different, I'll give you that. But I am curious... rather than selecting the folder and doing the copy to the flash drive, what happens in Win7 if you select all of the files (Ctrl-A) in the folder and copy them. Is the behavior the same? I am betting it isn't. And I think that may point out the difference in the two operating systems. Win7 is treating the folder as a single object and has no room to copy it. WinXP is treating the folder as a list of objects (like Win7 will do if you select all the files).

Sorry, but I think your method is ridiculous, copying 513mb just because a single 365kb file has changed. Waiting 2 minutes to do a job that should take 2 seconds if you used SyncToy. Why?

Gary

 

I can stand corrected on this, yes.

I've just been comfortable all these years doing it myself this way. I just don't like using tools like that. Maybe it's just me, but still......

Yes. I've tried that and it does the exact same thing. I can do this in XP (and prior OSes) as well without it barking at me.

That might be exactly what the difference is between the OSes as how XP treats it as a list while Vista & 7 treat it as a whole. Windows 7 does absolutely nothing different if I select all of the files and try to copy them that way. Same problem. XP lets me do it that way, too.

Well, it was a 365kb file that was actually ADDED rather than changed, but if I made changes to that file, it would replace the old with the new and that would work, too, to keep all files updated without having to go through each and every bloody file.

Furthermore, that one single 365kb file was an example. I have 50 folders in my WINTOOL folder I keep and add random amounts of files into those folders randomly... constantly. I am not going to try to keep track of all of those added files. If anybody had the patience for that, I'd say they were nuts.

If I did do them folder by folder I would have to sit at my computer the whole time copying and pasting files, waiting out the copy, starting another copy, wait out that copy, start another copy, etc. etc. rather than just copy the whole bloody folder, -> Overwrite -> Yes to all, --copy to take 8 min-- and walk away and do something else without having to constantly monitor it. Come back to my PC in 10 and Buh-Zing!!! Work is done for me.

btw, didn't you mention you had your sync tool give you a similar problem anyway? If that's the case and I were you, I'd be pissed and I'd just as well be questioning that.

I'm sorry if this is coming off wrong, ScuderiaConchiglia. You have actually been the most helpful here and seem to understand what I'm saying at least. I really do appreciate your patience with me. Regardless the opinions we may all have about things like this, I just hope there is a simple solution to this like a registry entry or even a hack to get around this.

 

Imagine you are copying 1GB worth of tree folder.
So the computer just issue a new copy of the 1GB worth of tree folder to maximize performance (People were complaining Vista was slow) then redirect all pointers continuously rather than checking a folder, copy in, redirect pointers, free up space.
Fragmentation is solvable by disk defrag BUT people were complaining about it as well...
It is like shopping market.
You don't buy 1 item then pay and bring home to be efficient, you buy all items together pay then bring home instead. Unless you are not strong enough to carry all of them.

If your files do not get updated selecting "No to all" when prompted to overwrite may resolve this.

 

I am not suggesting you go thru every file and folder, I am suggesting you stop the mass copy altogether and start using a proper tool for the job. A synch app like SyncToy.

The problem I had was because a SINGLE file was to big to be written in the available space and the tool I was using (not SyncToy, but another commercial product) would NOT overwrite the old file. And to be perfectly honest with you, for a single file I think that is the correct and proper behavior. Anything less can have leave the target folder in a state where the file is GONE, if there is some problem AFTER the original version that was there is deleted. (The tool I was using was designed for a server environment and you would not want to have a file disappear, having the wrong version of the file is much better than having no version at all!)

Don't worry, you are not coming off all wrong at all! This is a good discussion of the situation.

Gary

 

That does make perfect sense.

It was slow, but not necessarily because of this. If this were the case for Vista, it would have been the deal with all OSes. What people were complaining about Vista with and what really should have been being complained about were slightly different things. Vista was bad, but was given a worse reputation than it deserved because of absurd complaints.

That's what I mean about absurd complaints that people may have been giving. Doing a defrag in Windows 95, 98, ME and so on was more often just routine maintenance... like brushing your teeth!

How can someone complain about something like that and consider it a logical argument!? (Not anyone here directly, just anyone in general) I mean, really, how was Vista considered any slower than any other OS performing the same task. In this case with file copying.

That would be true in that situation, but with file copying and replacing, combining, adding..... in the computer world it is just different and cannot really be compared to RL situations in the same way.

I know what you're trying to say there, but if you look outside the box you would realize that this lacking function/feature/whichever is rather counter-productive.

I would do a lot of this, too! A LOT!!! Except, what I just showed you with Windows 7, it doesn't even let me get that far to click "No to all". Not to mention there is not that option, you only get a 'Yes to all' option. However, I would just hold the 'N' key on my keyboard to rapidly select no as each window would pop up and problem solved, but Windows 7 complains about disk space long before I even get that window. That still puts me back at square one.

edit:
I take that back about the 'no to all' option being missing. It is more accurately clicking the check-box for 'skip' and clicking 'do this for the next xxx items'. Nevertheless, it is still true that I cannot even get this far to even select such a thing.

 

Just for a closing on this thread, and for whoever may have looked here for an answer, I have found a workaround for this.

There is a program called "KillCopy" that you can download and install. It allows you to right click on the object you are copying and lets you select where you want for it to go. It will use its own interface and its own internal file management which completely bypasses windows file system allowing me to do exactly what I have been looking to do here. It is very small, lightweight, and does exactly what I want it to do. It's not too shabby and it even tells you how fast it is copying in whatever bytes/second, too. Pretty nice.

Anybody who has encountered this issue should at least try KillCopy if nothing else.

 

I have a problem that is in a way related to the OP's problem:
it also has to do with copy&pasting a file on a medium that has adequate free space, but still Windows says the file is too large.

To be more specific:
I have a 16GB usb memorystick (with 15GB free space) and am trying to copy an 8GB file to it, but the process cannot be completed because Windows indicates that the file is too large.

I've tried the KillCopy program but that only gets the job half done; exactly at 50% copying the same error pops up. Same problem on WinXP, Im on Vista.

In my case Im going to try another method:
Im going to archive and split the file into smaller parts (2 and a half hour process).

In the end its still a weird Windows problem.

 

That sir is correct. Most USB memory stick drives are formatted FAT, which has a 4 GB - 1 byte size limit.

 

Ah yes, that is correct. Thank you for the info.