The real and complete story - Does Windows defragment your SSD?
Sponsored By
There has been a LOT of confusion around Windows, SSDs (hard drives), and whether or not they are getting automatically defragmented by automatic maintenance tasks in Windows.
There's a general rule of thumb or statement that "defragging an SSD is always a bad idea." I think we can agree we've all heard this before. We've all been told that SSDs don't last forever and when they die, they just poof and die. SSDs can only handle a finite number of writes before things start going bad. This is of course true of regular spinning rust hard drives, but the conventional wisdom around SSDs is to avoid writes that are perceived as unnecessary.
I've seen statements around the web like this:
I just noticed that the defragsvc is hammering the internal disk on my machine. To my understanding defrag provides no value add on an SSD and so is disabled by default when the installer determines the disk is SSD. I was thinking it could be TRIM working, but I thought that was internal to the SSD and so the OS wouldn’t even see the IO.
One of the most popular blog posts on the topic of defrag and SSDs under Windows is by Vadim Sterkin. Vadim's analysis has a lot going on. He can see that defrag is doing something, but it's not clear why, how, or for how long. What's the real story? Something is clearly running, but what is it doing and why?
I made some inquiries internally, got what I thought was a definitive answer and waded in with a comment. However, my comment, while declarative, was wrong.
Windows doesn’t defrag SSDs. Full stop. If it reports as an SSD it doesn’t get defraged, no matter what. This is just a no-op message. There’s no bug here, sorry. - Me in the Past
I dug deeper and talked to developers on the Windows storage team and this post is written in conjunction with them to answer the question, once and for all
"What's the deal with SSDs, Windows and Defrag, and more importantly, is Windows doing the RIGHT THING?"
It turns out that the answer is more nuanced than just yes or no, as is common with technical questions.
The short answer is, yes, Windows does sometimes defragment SSDs, yes, it's important to intelligently and appropriately defrag SSDs, and yes, Windows is smart about how it treats your SSD.
The long answer is this.
Actually Scott and Vadim are both wrong. Storage Optimizer will defrag an SSD once a month if volume snapshots are enabled. This is by design and necessary due to slow volsnap copy on write performance on fragmented SSD volumes. It’s also somewhat of a misconception that fragmentation is not a problem on SSDs. If an SSD gets too fragmented you can hit maximum file fragmentation (when the metadata can’t represent any more file fragments) which will result in errors when you try to write/extend a file. Furthermore, more file fragments means more metadata to process while reading/writing a file, which can lead to slower performance.
As far as Retrim is concerned, this command should run on the schedule specified in the dfrgui UI. Retrim is necessary because of the way TRIM is processed in the file systems. Due to the varying performance of hardware responding to TRIM, TRIM is processed asynchronously by the file system. When a file is deleted or space is otherwise freed, the file system queues the trim request to be processed. To limit the peek resource usage this queue may only grow to a maximum number of trim requests. If the queue is of max size, incoming TRIM requests may be dropped. This is okay because we will periodically come through and do a Retrim with Storage Optimizer. The Retrim is done at a granularity that should avoid hitting the maximum TRIM request queue size where TRIMs are dropped.
Wow, that's awesome and dense. Let's tease it apart a little.
When he says volume snapshots or "volsnap" he means the Volume Shadow Copy system in Windows. This is used and enabled by Windows System Restore when it takes a snapshot of your system and saves it so you can rollback to a previous system state. I used this just yesterday when I install a bad driver. A bit of advanced info here - Defrag will only run on your SSD if volsnap is turned on, and volsnap is turned on by System Restore as one needs the other. You could turn off System Restore if you want, but that turns off a pretty important safety net for Windows.
One developer added this comment, which I think is right on.
I think the major misconception is that most people have a very outdated model of disk\file layout, and how SSDs work.
First, yes, your SSD will get intelligently defragmented once a month. Fragmentation, while less of a performance problem on SSDs vs traditional hard drives is still a problem. SSDS *do* get fragmented.
It's also worth pointing out that what we (old-timers) think about as "defrag.exe" as a UI is really "optimize your storage" now. It was defrag in the past and now it's a larger disk health automated system.
Additionally, there is a maximum level of fragmentation that the file system can handle. Fragmentation has long been considered as primarily a performance issue with traditional hard drives. When a disk gets fragmented, a singular file can exist in pieces in different locations on a physical drive. That physical drive then needs to seek around collecting pieces of the file and that takes extra time.
This kind of fragmentation still happens on SSDs, even though their performance characteristics are very different. The file systems metadata keeps track of fragments and can only keep track of so many. Defragmentation in cases like this is not only useful, but absolutely needed.
SSDs also have the concept of TRIM. While TRIM (retrim) is a separate concept from fragmentation, it is still handled by the Windows Storage Optimizer subsystem and the schedule is managed by the same UI from the User's perspective. TRIM is a way for SSDs to mark data blocks as being not in use. Writing to empty blocks on an SSD is faster that writing to blocks in use as those need to be erased before writing to them again. SSDs internally work very differently from traditional hard drives and don't usually know what sectors are in use and what is free space. Deleting something means marking it as not in use. TRIM lets the operating system notify the SSD that a page is no longer in use and this hint gives the SSD more information which results in fewer writes, and theoretically longer operating life.
In the old days, you would sometimes be told by power users to run this at the command line to see if TRIM was enabled for your SSD. A zero result indicates it is.
fsutil behavior query DisableDeleteNotify
However, this stuff is handled by Windows today in 2014, and you can trust that it's "doing the right thing." Windows 7, along with 8 and 8.1 come with appropriate and intelligent defaults and you don't need to change them for optimal disk performance. This is also true with Server SKUs like Windows Server 2008R2 and later.
Conclusion
No, Windows is not foolishly or blindly running a defrag on your SSD every night, and no, Windows defrag isn't shortening the life of your SSD unnecessarily. Modern SSDs don't work the same way that we are used to with traditional hard drives.
Yes, your SSD's file system sometimes needs a kind of defragmentation and that's handled by Windows, monthly by default, when appropriate. The intent is to maximize performance and a long life. If you disable defragmentation completely, you are taking a risk that your filesystem metadata could reach maximum fragmentation and get you potentially in trouble.
Related Links
* photo by Simon Wüllhorst, used under CC BY 2.0.
About Scott
Scott Hanselman is a former professor, former Chief Architect in finance, now speaker, consultant, father, diabetic, and Microsoft employee. He is a failed stand-up comic, a cornrower, and a book author.
About Newsletter
Hosting By
December 04, 2014 2:26
Huh.. Learn something new every day.
Ashley Grant
I still have one question. I manage a fleet of aging machines, and for the next refresh cycle we're considering just putting in a new power supply, maxing out the memory, and using Clonezilla to copy the disk to a new SSD. This addresses the main performance and reliability issues of the old PC, for about 1/5th the cost of a full replacement.
In this situation, where what is effectively *the same* Windows installation boots up and suddenly has an SSD where it didn't before, will Windows make the proper adjustments?
In this situation, where what is effectively *the same* Windows installation boots up and suddenly has an SSD where it didn't before, will Windows make the proper adjustments?
Funny, how often I've pondered this question and whether I needed to run trim or not. And could never find a definitive answer
Plus, I once tried striping a pair of SSDs years ago, hoping to get even higher levels of I/O but found out later that trim won't work on striped SSDs
Anyway, at the end of this short rambling. Thanks
Plus, I once tried striping a pair of SSDs years ago, hoping to get even higher levels of I/O but found out later that trim won't work on striped SSDs
Anyway, at the end of this short rambling. Thanks
Hi Scott,
Thanks for posting secret knowledge. I have a few comments on your and developer statements regarding my blog post you mentioned but didn't link to.
I can see defrag... defragging the drive with 5 passes, and I know for how long (obviously, YMMV depending on the drive and system specs). I also know why this is happening (the maintenance task forces defrag when the partition reaches the fragmentation threshold). Indeed, I didn't know why this has been implemented, but that was precisely my question to the product group!
What? :) This statement is poorly worded and doesn't make sense "as is". Care to clarify?
It's a regular defragmentation (the same one used for hard drives), and I don't see anything intelligent about it, except for being processed only as a part of the maintenance task.
The bottom line is that this gives yet another reason for people to turn off a very useful system protection feature. Until now they did it reduce writes to the disk. Now they'll also do it to avoid defrag. Myths are hard to kill...
Vadim
Thanks for posting secret knowledge. I have a few comments on your and developer statements regarding my blog post you mentioned but didn't link to.
While Vadim's analysis has a lot going on, I believe the analysis is still a little "black box." He can see that defrag is doing something, but it's not clear why, how, or for how long.
I can see defrag... defragging the drive with 5 passes, and I know for how long (obviously, YMMV depending on the drive and system specs). I also know why this is happening (the maintenance task forces defrag when the partition reaches the fragmentation threshold). Indeed, I didn't know why this has been implemented, but that was precisely my question to the product group!
Actually Scott and Vadim are both wrong. Storage Optimizer will defrag an SSD once a month if volume snapshots are enabled.Excuse me, but where am I wrong? :) I stated that with the default OS settings, "on a typical PC it occurs about once a month" (29.39 days on average based on the statistics I've collected from various PCs). I don't see other statements in this blog post that contradict to anything I've said, so either kindly ask the developer to clarify his or her statement or remove it altogether.
I think the major misconception is that most people have a very outdated model of disk\file layout, and how SSDs work.
What? :) This statement is poorly worded and doesn't make sense "as is". Care to clarify?
First, yes, your SSD will get intelligently defragmented once a month.
It's a regular defragmentation (the same one used for hard drives), and I don't see anything intelligent about it, except for being processed only as a part of the maintenance task.
The bottom line is that this gives yet another reason for people to turn off a very useful system protection feature. Until now they did it reduce writes to the disk. Now they'll also do it to avoid defrag. Myths are hard to kill...
Vadim
Hey Scott, Vadim,
Glad to get some more detail on this subject. It is good to know we weren't wrong in our observations that Windows was defragmenting SSDs, but it is useful to finally know why. Thank you Scott.
I guess I can go close off my thread on this subject on the Windows 10 forum!
Matt
Glad to get some more detail on this subject. It is good to know we weren't wrong in our observations that Windows was defragmenting SSDs, but it is useful to finally know why. Thank you Scott.
I guess I can go close off my thread on this subject on the Windows 10 forum!
Matt
Hi Scott, Would you also be able to shed light on the reason why compacting a VHDX file on a SSD fails to reduce the size? As a developer I have a SSD on my Windows 8.1 laptop with client Hyper-V. I have several VHDX files on the SSD. Inside the VM a hard disk reports it uses for example 20GB, but the VHDX is 30GB. Compacting the VHDX fails to reduce the size to 20GB. Not sure if running degrag / optimize on the VHDX on a SSD is a good idea, but it doesn't seem to make a difference regardless.
Further to that: where the host OS may be clever enough to know it's running on a SSD and how to optimize it, does a guest OS in a VM running on a VHDX on a SSD also know it is running on a SSD and be clever how to optimise it?
@Vadim, I think the "Vadim is wrong" comes from your article that states "yet there is a bug in Windows 8+", which according to the product guys is clearly not the case.
@John Rothlisberger
It's easy to be correct in hindsight and operate terms like "clearly" :) You have to take into account that I had posted my article almost two years before this explanation was made public. And until this point the product team was either mum or completely denied SSD defrag via Scott (although I have no idea whether it was the same team).
From the QA perspective, such an abnormal product behavior is clearly a bug, unless it's documented. So it has to be raised and then the dev team should go ahead and explain why this is not a bug. The product team has failed to document this behavior in the first place, and then Microsoft kept ignoring reports raised on its official forums (yes, they are officially bug reports, at least on Answers) and via MS Connect. This is what really wrong here.
And I'll certainly test whether the volsnap explanation is correct. It's easy, just takes time. I mean use your critical thinking and don't believe everything MSFT employees tell you, because they can be totally wrong (and this blog post is a great example of this :)
Vadim
It's easy to be correct in hindsight and operate terms like "clearly" :) You have to take into account that I had posted my article almost two years before this explanation was made public. And until this point the product team was either mum or completely denied SSD defrag via Scott (although I have no idea whether it was the same team).
From the QA perspective, such an abnormal product behavior is clearly a bug, unless it's documented. So it has to be raised and then the dev team should go ahead and explain why this is not a bug. The product team has failed to document this behavior in the first place, and then Microsoft kept ignoring reports raised on its official forums (yes, they are officially bug reports, at least on Answers) and via MS Connect. This is what really wrong here.
And I'll certainly test whether the volsnap explanation is correct. It's easy, just takes time. I mean use your critical thinking and don't believe everything MSFT employees tell you, because they can be totally wrong (and this blog post is a great example of this :)
Vadim
Hi @Vadim, Scott's and yours were both very interesting posts. But just because Microsoft hasn't commented or documented something doesn't make it a bug. Even though I am a very skeptical person and everything must pass my logic test, I believe that great companies like Microsoft and Apple put a lot of thought into most things they do (which is why for example I am a strong believer in leaving defaults alone as much as possible, because I trust that significant consideration went into what those defaults should be in the first place). Think about the discussion that happened at some point inside Microsoft about how SSDs should be handled... We all know defragmenting an SSD is unnecessary and bad for the drive as it wears out the limited write cycles. Except we've now learned that there are certain file system limits (something I was not aware of), like a maximum number of fragments. I don't think you are saying that Microsoft developers never had that discussion and that this is a bug and that SSDs should never ever be defragmented. Defragmenting an SSD once a month is not abnormal if that is what Microsoft intended.
I don't follow why "this blog post is a great example of this".
I hope all that makes sense, and thanks for triggering this discussion in the first place.
I don't follow why "this blog post is a great example of this".
I hope all that makes sense, and thanks for triggering this discussion in the first place.
@John Rothlisberger
Well, of course only a dev or PM can say whether it's a bug or not, because they have access to the internal documentation. That's why the company must explain such controversial behavior of its product beforehand, and not two years later (and even now the product team delivers the message thru this blog instead of publishing the official KB article stating that this behavior is by design).
And all MSFT has claimed until today is that Windows does not defragment SSD. But if this statement was true, then defrag was a bug :) And only since today it's not a bug for outsiders.
Now you come and talk about significant considerations for defaults and FS limitations you didn't know about. Think of Windows 7 then. It doesn't defrag SSD and never has, but it is subject to the same FS limitations. Please read the first paragraph of the product team message and apply it to Windows 7 on SSD. Suddenly, it can be affected by the issues they describe :) And what have they done to prevent them?
А Microsoft employee publicly states that Windows doesn't defrag SSDs using full stops, then he reiterates his point citing some product team, and now finally states that he was wrong. I guess if Scott looked into the evidence before making his claims, he'd come up with the correct answer sooner.
Well, of course only a dev or PM can say whether it's a bug or not, because they have access to the internal documentation. That's why the company must explain such controversial behavior of its product beforehand, and not two years later (and even now the product team delivers the message thru this blog instead of publishing the official KB article stating that this behavior is by design).
And all MSFT has claimed until today is that Windows does not defragment SSD. But if this statement was true, then defrag was a bug :) And only since today it's not a bug for outsiders.
Now you come and talk about significant considerations for defaults and FS limitations you didn't know about. Think of Windows 7 then. It doesn't defrag SSD and never has, but it is subject to the same FS limitations. Please read the first paragraph of the product team message and apply it to Windows 7 on SSD. Suddenly, it can be affected by the issues they describe :) And what have they done to prevent them?
I don't follow why "this blog post is a great example of this".
А Microsoft employee publicly states that Windows doesn't defrag SSDs using full stops, then he reiterates his point citing some product team, and now finally states that he was wrong. I guess if Scott looked into the evidence before making his claims, he'd come up with the correct answer sooner.
@Vadim
Does Microsoft categorically state they do not defragment SSDs?
Does Microsoft categorically state they do not defragment SSDs?
"SSDs don't last forever and when they die, they just poof and die"
I've read that for SSD's, when a block can be longer written to, it can be set to read-only, thus keeping your data intact. Some systems may actually copy the data to a new writable block and tag the read-only block as unusable. In any case, your SSD doesn't suddenly poof and die, it slowly degrades/shrinks at the end of its life-cycle, but it doesn't have the catastrophic effect implied by your statement. I think this is a subtle distinction worth making. Somebody correct me if I'm wrong.
I've read that for SSD's, when a block can be longer written to, it can be set to read-only, thus keeping your data intact. Some systems may actually copy the data to a new writable block and tag the read-only block as unusable. In any case, your SSD doesn't suddenly poof and die, it slowly degrades/shrinks at the end of its life-cycle, but it doesn't have the catastrophic effect implied by your statement. I think this is a subtle distinction worth making. Somebody correct me if I'm wrong.
John - I research this issue earlier, got bad information, and stated that it does not defrag SSDs. This blog post is my correction. Certainly it doesn't Defrag them the same way, or with the same frequency, but it does once a month, and with good reason.
@John Rothlisberger
What's you point exactly? That I shouldn't use the word "bug", because I don't know for sure? This is what bug trackers are for - you file a bug and find out when you get a response. This is exactly what I did on Connect, but got nowhere. This is what people did on TechNet and Answers and got nowhwere.
To answer your question specifically: The last official statement was that Windows does not defragment SSDs, but it applied to Windows 7.
@Scott Hanselman
Where exactly does this come from? Are you hiding more secret knowledge? :)
Does Microsoft categorically state they do not defragment SSDs?c
What's you point exactly? That I shouldn't use the word "bug", because I don't know for sure? This is what bug trackers are for - you file a bug and find out when you get a response. This is exactly what I did on Connect, but got nowhere. This is what people did on TechNet and Answers and got nowhwere.
To answer your question specifically: The last official statement was that Windows does not defragment SSDs, but it applied to Windows 7.
@Scott Hanselman
Certainly it doesn't Defrag them the same way
Where exactly does this come from? Are you hiding more secret knowledge? :)
@Dan Bailiff
I guess you mean JDEC specifications, or End of Life Data Retention of 1 Year in manufacturers' terms.
In practice, I haven't seen a single endurance test with the purpose to exhaust NAND resource that would demonstrate this. They just die :) But this happens well after total host writes bypass the threshold guaranteed by the drive manufacturer.
I guess you mean JDEC specifications, or End of Life Data Retention of 1 Year in manufacturers' terms.
In practice, I haven't seen a single endurance test with the purpose to exhaust NAND resource that would demonstrate this. They just die :) But this happens well after total host writes bypass the threshold guaranteed by the drive manufacturer.
If I'm following what it's doing properly, I'd still argue that an SSD still doesn't benefit from defrag the way a traditional hard drive does (in reducing number of seeks). It's the Windows filesystem (assumed here to be NTFS) that can only track a limited number of extents and needs the defrag. If the FS weren't so limited, the SSD would happily keep running with files as fragmented as you please, without the performance drop seen on traditional hard drives.
@Joshua Ochs
You're correct. The whole OS↔SSD interaction is built on legacy HDD principles. The OS writes to LBAs, and marks them free when you delete files. The SSD has no idea about this, and that's why it needs TRIM to learn that these LBAa are free.
On the other hand, the OS has no idea how the SSD controller maps LBAs to physical NAND cells. It optimizes data placement using Garbage Collection / NAND defrag algorithms. So when you defrag, you simply optimize the FS, but you also force the controller to perform read-modify-write operations, and it moves data around NAND cells... aimlessly (from its standpoint, if it had one :)
I don't know what intelligent defrag Scott is talking about, but the second part of the product team message is about a different operation, retrim. And it's called retrim, because TRIM commands have already been sent at deletion time. The OS is simply telling the SSD controller: I've sent you a bunch of mail, did you get it all? But again, it's not an order, and it's up to the controller to decide what to do with this mail. Maybe, it's all spam :)
You're correct. The whole OS↔SSD interaction is built on legacy HDD principles. The OS writes to LBAs, and marks them free when you delete files. The SSD has no idea about this, and that's why it needs TRIM to learn that these LBAa are free.
On the other hand, the OS has no idea how the SSD controller maps LBAs to physical NAND cells. It optimizes data placement using Garbage Collection / NAND defrag algorithms. So when you defrag, you simply optimize the FS, but you also force the controller to perform read-modify-write operations, and it moves data around NAND cells... aimlessly (from its standpoint, if it had one :)
I don't know what intelligent defrag Scott is talking about, but the second part of the product team message is about a different operation, retrim. And it's called retrim, because TRIM commands have already been sent at deletion time. The OS is simply telling the SSD controller: I've sent you a bunch of mail, did you get it all? But again, it's not an order, and it's up to the controller to decide what to do with this mail. Maybe, it's all spam :)
Following the whole saga, it is disingenuous and stupid for anyone at Microsoft to claim that vigilantes like Vadim are wrong.
This takes Microsoft down a notch on the credibility scale, yet again.
This takes Microsoft down a notch on the credibility scale, yet again.
I didn't mean to imply an SSD-special defrag, sorry about that, just that it knows they are SSDs and the scheduling is different based on that. Also, one other thing to add, the dev told me (OS related, not HW related) that could be useful:
On server we disabled all but slab consolidation and tier optimization for thinly provisioned volumes and tiered volumes respectively.
@Scott Hanselman
1. All volumes are processed by the same ScheduledDefrag task triggered by Automatic Maintenance (AM), and only under this condition SSD defrag may occur naturally. But this task has no visible triggers (XML shows that AM runs it), and there's no other defrag task with the monthly schedule. If the PC is never idle for several days in a row, it may not enter the maintenance mode in exactly one month.
2. In my experiments, I haven't been able to reproduce defrag when the volume fragmentation was below 10%, but saw defrag at 11%. There must be a fragmentation threshold, and it makes sense - why would you defrag a non-fragmented drive? I believe the threshold is the same for SSDs and HDDs, but only the product team knows for sure.
All this dev language and some of your off-target explanations just make it harder to see a simple fact: Windows out of the box defragments HDDs and SSDs on equal terms using the same scheduled task and [probably] the same fragmentation threshold. Yet it doesn't expose any options to perform SSD defrag, with the exception of the scheduled task.
I think the best way to get The real and complete story is to have the product team explain the behavior clearly (ideally, document it in MSKB). While I do appreciate your efforts on trying to clear this up (at least one person at MSFT is trying!), it starts to look more and more like Chinese whispers.
Vadim
P.S. Strange, all comments are gone now. I hope they will be back soon.
just that it knows they are SSDs and the scheduling is different based on thatThe scheduling is not different. And I believe once a month in the product team quote is an approximation for two reasons.
1. All volumes are processed by the same ScheduledDefrag task triggered by Automatic Maintenance (AM), and only under this condition SSD defrag may occur naturally. But this task has no visible triggers (XML shows that AM runs it), and there's no other defrag task with the monthly schedule. If the PC is never idle for several days in a row, it may not enter the maintenance mode in exactly one month.
2. In my experiments, I haven't been able to reproduce defrag when the volume fragmentation was below 10%, but saw defrag at 11%. There must be a fragmentation threshold, and it makes sense - why would you defrag a non-fragmented drive? I believe the threshold is the same for SSDs and HDDs, but only the product team knows for sure.
All this dev language and some of your off-target explanations just make it harder to see a simple fact: Windows out of the box defragments HDDs and SSDs on equal terms using the same scheduled task and [probably] the same fragmentation threshold. Yet it doesn't expose any options to perform SSD defrag, with the exception of the scheduled task.
I think the best way to get The real and complete story is to have the product team explain the behavior clearly (ideally, document it in MSKB). While I do appreciate your efforts on trying to clear this up (at least one person at MSFT is trying!), it starts to look more and more like Chinese whispers.
Vadim
P.S. Strange, all comments are gone now. I hope they will be back soon.
Vadim - Some thing happened and I've lost comments on the last TWO blog posts. I'm working on getting them restored by my ISP.
According to the devs I talked to, the schedule for SSDs is every 28 days. Specifically the scheduled task checks every week if the SSD has been defragged in the last 28 days, and if it hasn't it defrags. If not, it checks again next week.
According to the devs I talked to, the schedule for SSDs is every 28 days. Specifically the scheduled task checks every week if the SSD has been defragged in the last 28 days, and if it hasn't it defrags. If not, it checks again next week.
Scott, thanks for providing additional info you should've supplied in the original post. This just confirms that "once a month" is an approximation. Ok, they just ensure the SSD defrag period is less than 28 days.
But there's no word on the fragmentation level (you haven't discussed or disclosed it), so now it sounds like they defrag SSD regardless of how badly it is fragmented. If I had barely used my PC within the last month, it won't get fragmented enough to justify defrag. That's even less intelligent :)
That's why I'm saying that the product team must deliver an elaborate and clear message that explains why they have issues with volsnap write-on-copy performance and how exactly SSD defrag is implemented. This is the only way to end misleading interpretations.
As for your lost comments, I have a copy of the entire page before the loss http://1drv.ms/1u2YYyF (it'll be public until you restore comments)
FWIW, I backup my blog DB daily, including comments.
Vadim
But there's no word on the fragmentation level (you haven't discussed or disclosed it), so now it sounds like they defrag SSD regardless of how badly it is fragmented. If I had barely used my PC within the last month, it won't get fragmented enough to justify defrag. That's even less intelligent :)
That's why I'm saying that the product team must deliver an elaborate and clear message that explains why they have issues with volsnap write-on-copy performance and how exactly SSD defrag is implemented. This is the only way to end misleading interpretations.
As for your lost comments, I have a copy of the entire page before the loss http://1drv.ms/1u2YYyF (it'll be public until you restore comments)
FWIW, I backup my blog DB daily, including comments.
Vadim
Funny... The Spanish edition of the "Optimize Drives" dialogue translates "OK" (correct) as "Aceptar" (accept), which of course makes little sense.
Thanks Vadim - This blog doesn't use a DB so it's a little more tedious to FTP all the updated comments. My ISP has a backup, I just need to see how much I lost, and optionally go manually add all the comments back. Ick.
An SSD is:
a bunch of NAND die
in a bunch of packages
wired through a bunch of parallel channels
to a controller (SandForce, Marvell, Intel, etc.) which may compress by design (SandForce, if only)
I'd wager that SSD controllers are vastly different, one from another, in how they read/write the NAND than HDD controllers, which support the current generations of platter and head technology.
That controller, implementing multiple channels (as many as 32 in even prosumer drives) to the multiple die under various rubrics (RAISE in SandForce speak), moves the data around as needed for wear leveling. Some controllers level more aggressively than others, but the basic structure remains the same: there is no, on purpose, contiguous (i.e., sequential) storage in an SSD. The bits are strewn over available dies by the controller in the way it finds most efficient.
The extent to which sequential reads appear faster than random reads is due to SSD resident caching (sometimes DRAM, sometimes NAND) and read-ahead, not defragging.
My guess, and it's only that since I don't write Windows code, is that the defrag process only gets as far as the FTL in the SSD controller. As such, the NAND die never see the process.
a bunch of NAND die
in a bunch of packages
wired through a bunch of parallel channels
to a controller (SandForce, Marvell, Intel, etc.) which may compress by design (SandForce, if only)
I'd wager that SSD controllers are vastly different, one from another, in how they read/write the NAND than HDD controllers, which support the current generations of platter and head technology.
That controller, implementing multiple channels (as many as 32 in even prosumer drives) to the multiple die under various rubrics (RAISE in SandForce speak), moves the data around as needed for wear leveling. Some controllers level more aggressively than others, but the basic structure remains the same: there is no, on purpose, contiguous (i.e., sequential) storage in an SSD. The bits are strewn over available dies by the controller in the way it finds most efficient.
The extent to which sequential reads appear faster than random reads is due to SSD resident caching (sometimes DRAM, sometimes NAND) and read-ahead, not defragging.
My guess, and it's only that since I don't write Windows code, is that the defrag process only gets as far as the FTL in the SSD controller. As such, the NAND die never see the process.
I'm even more confused now.
I followed Vadim's instructions to disable defragging.
So are you saying that Vadim is incorrect and following his instructions can be bad for my system?
If either you or Vadim can clarify, that would be appreciated. Thanks!
I followed Vadim's instructions to disable defragging.
So are you saying that Vadim is incorrect and following his instructions can be bad for my system?
If either you or Vadim can clarify, that would be appreciated. Thanks!
@Vadim i think you are right regarding the fact that some divisions at Microsoft tend to answer possible bug reports with... nothing. MS was never good at communication. Always talking about user feedback but ignoring it. There should be a KB Article as so many people are confused.
@Scott Thank you for working hard on talking to all the folks outside of MSFT. Guess we all know that you can't say anything you know, as there are always some political side effects in such a big company but at least you try to get some concrete info for us.
I really hope that all MS Divisions will some day realize that ignoring customers questions and feedback will cause a loss of customers as soon as there are good alternatives from other vendors. This is not only about windows. This is about every product.
I don't say they have to implement everything. I also believe that all the default settings and implementations have some kind of good reason.
But in situations like this where strange behaviors can bee seen and measured, not talking to the folks outside lets me think: They have no idea of why this is happening.
Don't believe like this helps people trusting in a software vendor.
Anyway: Thank you both so much for trying to give us all some input!
@Scott Thank you for working hard on talking to all the folks outside of MSFT. Guess we all know that you can't say anything you know, as there are always some political side effects in such a big company but at least you try to get some concrete info for us.
I really hope that all MS Divisions will some day realize that ignoring customers questions and feedback will cause a loss of customers as soon as there are good alternatives from other vendors. This is not only about windows. This is about every product.
I don't say they have to implement everything. I also believe that all the default settings and implementations have some kind of good reason.
But in situations like this where strange behaviors can bee seen and measured, not talking to the folks outside lets me think: They have no idea of why this is happening.
Don't believe like this helps people trusting in a software vendor.
Anyway: Thank you both so much for trying to give us all some input!
@Charles:
-- I'm even more confused now.
The re-appeared comments (not here when I submitted) should help.
-- I'm even more confused now.
The re-appeared comments (not here when I submitted) should help.
@Vadim:
-- And it's called retrim, because TRIM commands have already been sent at deletion time.
It's generally accepted that one shouldn't set the 'trim on deletion' option if available, since this just write amplifies big time. TRIM every week or so should be sufficient.
Also, the implementation of a SSD controller need not demand that NAND cells be re-written when a defrag call from the OS hits the controller. The controller code developer had/has the option to reorder the FTL (LBAs), keeping the NAND cell addresses attached to the FTL entries. No cell data has to be rewritten. Although a specific controller may do that.
-- And it's called retrim, because TRIM commands have already been sent at deletion time.
It's generally accepted that one shouldn't set the 'trim on deletion' option if available, since this just write amplifies big time. TRIM every week or so should be sufficient.
Also, the implementation of a SSD controller need not demand that NAND cells be re-written when a defrag call from the OS hits the controller. The controller code developer had/has the option to reorder the FTL (LBAs), keeping the NAND cell addresses attached to the FTL entries. No cell data has to be rewritten. Although a specific controller may do that.
@Max
If you're confused, you shouldn't change the default OS settings, and in this case you should accept SSD defrag.
If you want to understand under what conditions you may experience reduced performance, read the product team explanation
What they tell you is that they have a performance issue when writing volume snapshots on a fragmented SSD, so they use defrag to work around the issue. For a home PC, it's pretty much about creating restore points. Now think when they are created and whether it matters to you.
The rest of their explanation has nothing to do with the reason they have implemented SSD defrag, because
@Robert Young
Per the product team quote, they queue TRIM commands and process them asynchronously, not once a week. They follow up with these commands using the ScheduledDefrag task.
Use TRIMcheck to see how fast TRIM commands are processed under Windows. It's a matter of seconds, unless your drive is lazy or have an issue of some sort.
If you're confused, you shouldn't change the default OS settings, and in this case you should accept SSD defrag.
If you want to understand under what conditions you may experience reduced performance, read the product team explanation
necessary due to slow volsnap copy on write performance on fragmented SSD volumes
What they tell you is that they have a performance issue when writing volume snapshots on a fragmented SSD, so they use defrag to work around the issue. For a home PC, it's pretty much about creating restore points. Now think when they are created and whether it matters to you.
The rest of their explanation has nothing to do with the reason they have implemented SSD defrag, because
Storage Optimizer will defrag an SSD once a month if volume snapshots are enabled
@Robert Young
It's generally accepted that one shouldn't set the 'trim on deletion' option if available, since this just write amplifies big time. TRIM every week or so should be sufficient.Where does "generally accepted" and "every week" come from? Delaying TRIM commands for a week leads to reduced drive performance, especially when it gets close to being full. Since you don't tell the drive that the LBA is free, it can't use this info during GC. It learns about free LBAs upon OS write operations in these LBAs, and has to perform read-modify-write operations instead of simply writing to these LBAs.
Per the product team quote, they queue TRIM commands and process them asynchronously, not once a week. They follow up with these commands using the ScheduledDefrag task.
Use TRIMcheck to see how fast TRIM commands are processed under Windows. It's a matter of seconds, unless your drive is lazy or have an issue of some sort.
@Vadim:
If weekly isn't often enough, then daily. 'on delete' buys you just more write amplification. If a SSD is so full that the only way to get clean blocks is TRIM 'on delete', then a bigger drive is called for.
If weekly isn't often enough, then daily. 'on delete' buys you just more write amplification. If a SSD is so full that the only way to get clean blocks is TRIM 'on delete', then a bigger drive is called for.
@Robert Young
Actually, it's totally the opposite :) The sooner the controller knows about empty LBAs, the better. That way it doesn't move this data around during GC, if it doesn't have to or doesn't "think" it's the best time to do it. This reduces WA.
Like I said, try TRIMCheck on Windows and Linux to see how different OSs implement TRIM and how long it takes SSD controllers to react to deleted data (if you prefer hardcore, just use a disk editor). Then think why they react so fast, while they could wait another day or week (sic!).
Vadim
'on delete' buys you just more write amplification.
Actually, it's totally the opposite :) The sooner the controller knows about empty LBAs, the better. That way it doesn't move this data around during GC, if it doesn't have to or doesn't "think" it's the best time to do it. This reduces WA.
Like I said, try TRIMCheck on Windows and Linux to see how different OSs implement TRIM and how long it takes SSD controllers to react to deleted data (if you prefer hardcore, just use a disk editor). Then think why they react so fast, while they could wait another day or week (sic!).
Vadim
@Robert Young
SSDs treat TRIM commands as 'suggestions', not orders. TRIMming some sectors will invalidate the underlying data (and signal this to the OS) and internally mark corresponding NAND pages as free to use for SSD wear leveling operations, but actual deletion from NAND might occur later in time, if even at all, depending on user usage patterns.
If TRIM caused certain and immediate deletion of data from the SSD, there wouldn't be need for the Secure Erase command to wipe a drive.
SSDs treat TRIM commands as 'suggestions', not orders. TRIMming some sectors will invalidate the underlying data (and signal this to the OS) and internally mark corresponding NAND pages as free to use for SSD wear leveling operations, but actual deletion from NAND might occur later in time, if even at all, depending on user usage patterns.
If TRIM caused certain and immediate deletion of data from the SSD, there wouldn't be need for the Secure Erase command to wipe a drive.
@SSBrain
I'm seeing a mix of solid and shaky statements.
Vadim
I'm seeing a mix of solid and shaky statements.
TRIMming some sectors will invalidate the underlying data (and signal this to the OS)What do you mean by signaling to the OS? The OS deletes a file → marks LBAs as free → sends a TRIM command to the controller. The OS already knows these LBAs are free, and the controller doesn't tell the OS anything about them.
If TRIM caused certain and immediate deletion of data from the SSD, there wouldn't be need for the Secure Erase command to wipe a drive.Generally, there's no real need for secure erase. Regular quick formatting takes care of it. Yes, there are some TRIM issues with SF controllers that formatting may not take care of, but they only surface under extreme conditions (AnandTech calls them torture for a reason :) Test labs use SE as a baseline.
Vadim
@Vadim
What I mean is that even if the controller returns '0' after performing read operations over a TRIM'd LBA range (which is what TRIMCheck looks for), there's no guarantee that the underlying data was actually deleted, only that it might happen at some point in the future for wear leveling/internal housekeeping operations. In other words, when this happens, data has been invalidated and made inaccessible by standard means, but not necessarily (immediately) deleted.
The TRIM command which causes this behavior is called Deterministic Read Zero after Trim (DZAT) and is defined in the SATA protocol. Windows uses this kind of TRIM command by default, which is what most (but not all) modern SSDs support.
See here for more detailed information: http://ru.belkasoft.com/en/ssd-2014
What I mean is that even if the controller returns '0' after performing read operations over a TRIM'd LBA range (which is what TRIMCheck looks for), there's no guarantee that the underlying data was actually deleted, only that it might happen at some point in the future for wear leveling/internal housekeeping operations. In other words, when this happens, data has been invalidated and made inaccessible by standard means, but not necessarily (immediately) deleted.
The TRIM command which causes this behavior is called Deterministic Read Zero after Trim (DZAT) and is defined in the SATA protocol. Windows uses this kind of TRIM command by default, which is what most (but not all) modern SSDs support.
See here for more detailed information: http://ru.belkasoft.com/en/ssd-2014
Will TRIM still work monthly if I have system restore disabled?
@SSBrain
Thanks for a solid explanation. It's always fascinating to read about SSD forensics. There's also an interesting note in this article:
@T3h Ub3r K1tten
Yes, it will, and probably more often. Check out event 258 in your logs.
Vadim
Thanks for a solid explanation. It's always fascinating to read about SSD forensics. There's also an interesting note in this article:
Notably, DRAT is not implemented in Windows, as NTFS does not allow applications reading the trimmed data.
@T3h Ub3r K1tten
Yes, it will, and probably more often. Check out event 258 in your logs.
Vadim
I have an SSD that only contains a number of VMs. Is there any reason to defrag the host SSD, or to defrag the drive 'inside' the client Windows systems?
While Microsoft has been talking publicly for years about the Windows storage subsystem, the info isn't always readily available. I agree with Vadim that Microsoft could do a *much* better job of disseminating this info.
A couple of articles I found enlightening: Storage Conference Presentation, TechNet article. (And, of course, this blog and thread.)
From what I can tell, Microsoft has modified the existing Defrag tool, so that it does double-duty. It performs an old-fashioned defrag on platter drives, and it does something different on SSDs. I can easily see why people think that Windows is defragmenting their SSDs.
And it probably is, to some extent. Microsoft says that the file system can only handle a certain amount of fragments. I am curious as to what the limits are, and how I can query the current state of my systems. And does it always defrag the file system on an SSD, or does it only do it if a certain limit is exceeded? (I suspect the latter, but ... without a KB article on it, who knows?)
As for the ReTRIM commands, my understanding is that the TRIM commands are sent every time from the OS to the SSD. However, there is no guarantee that all of the TRIM operations were successfully executed. It's easy enough for me to see where a "very busy" system might overload a buffer on the SSD, and some of the TRIM operations might not occur. So, from what I've read, it looks like one of the things that the "monthly" Optimize Drives will do is to send new TRIM commands (ReTRIM) for all the free space on the drive. Since this should only happen during idle time, the drive should then be able to execute these TRIM operations and free up all of the free space.
I'm not sure what the limits are on modern Flash chips. It's at least 1,000,000 P/E on newer chips (Wikipedia article). I'm guessing that a once-a-month Optimize operation is not putting undue wear-and-tear on the SSD.
(Even if each Optimize operation resulted in 10 writes to each and every block on the Flash chips, that's only 600 P/E cycles over 5 years. Not that there's any reason to think it does something like 10 writes to every block; I'm just making up a worst-case scenario.)
I'm also curious as to how Windows monitors the health of an SSD, if at all. Scott wrote "We've all been told that SSDs don't last forever and when they die, they just poof and die." But this is not true. (OK, it's true that some people go around saying it. *grin*) But the SSDs have extra storage space built inside. As cells wear out, the SSD replaces them with spare cells. I've seen SSD specs where the drive size was 3/4 of the total space inside the SSD (so the SSD had 33% more space inside than you would have thought). The internal SSD controller knows how many bad cells there are, and it can (or should) be able to report how close it is to running out. Storage arrays (I'm a SAN engineer) monitor the health of their SSDs, and call for pro-active replacement as the drives start having problems. It would be nice to see Windows doing something similar when it connects to SSDs.
A couple of articles I found enlightening: Storage Conference Presentation, TechNet article. (And, of course, this blog and thread.)
From what I can tell, Microsoft has modified the existing Defrag tool, so that it does double-duty. It performs an old-fashioned defrag on platter drives, and it does something different on SSDs. I can easily see why people think that Windows is defragmenting their SSDs.
And it probably is, to some extent. Microsoft says that the file system can only handle a certain amount of fragments. I am curious as to what the limits are, and how I can query the current state of my systems. And does it always defrag the file system on an SSD, or does it only do it if a certain limit is exceeded? (I suspect the latter, but ... without a KB article on it, who knows?)
As for the ReTRIM commands, my understanding is that the TRIM commands are sent every time from the OS to the SSD. However, there is no guarantee that all of the TRIM operations were successfully executed. It's easy enough for me to see where a "very busy" system might overload a buffer on the SSD, and some of the TRIM operations might not occur. So, from what I've read, it looks like one of the things that the "monthly" Optimize Drives will do is to send new TRIM commands (ReTRIM) for all the free space on the drive. Since this should only happen during idle time, the drive should then be able to execute these TRIM operations and free up all of the free space.
I'm not sure what the limits are on modern Flash chips. It's at least 1,000,000 P/E on newer chips (Wikipedia article). I'm guessing that a once-a-month Optimize operation is not putting undue wear-and-tear on the SSD.
(Even if each Optimize operation resulted in 10 writes to each and every block on the Flash chips, that's only 600 P/E cycles over 5 years. Not that there's any reason to think it does something like 10 writes to every block; I'm just making up a worst-case scenario.)
I'm also curious as to how Windows monitors the health of an SSD, if at all. Scott wrote "We've all been told that SSDs don't last forever and when they die, they just poof and die." But this is not true. (OK, it's true that some people go around saying it. *grin*) But the SSDs have extra storage space built inside. As cells wear out, the SSD replaces them with spare cells. I've seen SSD specs where the drive size was 3/4 of the total space inside the SSD (so the SSD had 33% more space inside than you would have thought). The internal SSD controller knows how many bad cells there are, and it can (or should) be able to report how close it is to running out. Storage arrays (I'm a SAN engineer) monitor the health of their SSDs, and call for pro-active replacement as the drives start having problems. It would be nice to see Windows doing something similar when it connects to SSDs.
Wow, people sure are jerks.
First Scott gets asked about SSDs and people whine there is no information, so he asks some people around the office and gets a largely correct answer and disseminates that information as best he can.
Then someone else sees other facts and writes up what THEY know and what THEY suspect.
Then Scott dives further and gets more accurate (and possibly even different information if things have changed since his first asking) and disseminates that as a "correction" post.
Then someone else whines that Microsoft never answers things.
Then someone else whines that Scott is answering what he doesn't KNOW.
Then first person gets all in a huff about the language used by someone Scott quoted and defends themselves as merely filing a bug report to get the correct information.
Then that person whines that the newer information should have been in the original post in the first place.
If I was Scott, I just wouldn't bother with you lot.
First Scott gets asked about SSDs and people whine there is no information, so he asks some people around the office and gets a largely correct answer and disseminates that information as best he can.
Then someone else sees other facts and writes up what THEY know and what THEY suspect.
Then Scott dives further and gets more accurate (and possibly even different information if things have changed since his first asking) and disseminates that as a "correction" post.
Then someone else whines that Microsoft never answers things.
Then someone else whines that Scott is answering what he doesn't KNOW.
Then first person gets all in a huff about the language used by someone Scott quoted and defends themselves as merely filing a bug report to get the correct information.
Then that person whines that the newer information should have been in the original post in the first place.
If I was Scott, I just wouldn't bother with you lot.
@Jim Burd
Windows monitors disk health using SMART info collected with scheduled tasks residing in Microsoft - Windows - DiskDiagnostics. I have no idea about the specifics, but in an ideal world, it would tell you that your drive is reaching the threshold (e.g. Wear Leveling Count).
@Marc Brooks
You have to get the correct sequence of events before calling other people jerks.
Scott gets asked on twitter → comes to my blog and makes an authoritative but false statement based on his beliefs → gets more pressure on twitter → contacts some product team → states the team confirms no defrag
Scott gets asked on twitter again → contacts another team → gets a different answer → posts this blog entry where he admits he was wrong → provides correct, but incomplete info → gets more pressure → reveals more info
There's no whining here, except for your attacks. I'm only interested in technical discussion. If Scott doesn't know anything about technical details on this topic, perhaps, he shouldn't be posting false statements on other blogs without checking the facts as well as delivering incomplete info on his blog post called "complete story".
Vadim
Windows monitors disk health using SMART info collected with scheduled tasks residing in Microsoft - Windows - DiskDiagnostics. I have no idea about the specifics, but in an ideal world, it would tell you that your drive is reaching the threshold (e.g. Wear Leveling Count).
@Marc Brooks
You have to get the correct sequence of events before calling other people jerks.
Scott gets asked on twitter → comes to my blog and makes an authoritative but false statement based on his beliefs → gets more pressure on twitter → contacts some product team → states the team confirms no defrag
Scott gets asked on twitter again → contacts another team → gets a different answer → posts this blog entry where he admits he was wrong → provides correct, but incomplete info → gets more pressure → reveals more info
There's no whining here, except for your attacks. I'm only interested in technical discussion. If Scott doesn't know anything about technical details on this topic, perhaps, he shouldn't be posting false statements on other blogs without checking the facts as well as delivering incomplete info on his blog post called "complete story".
Vadim
@Vadim
Interesting. I have several drives that are in various states of ill-health. I monitor them via Acronis Disk Monitor. The stats returned by ADM match those of other SMART tools I use. They are all currently connected to Win7 physical servers. I don't recall ever seeing any pro-active messages from Windows about these drives.
My Win8.1 and Win10 systems are all VMs. I'll have to connect some of these bad drives to the newer systems to see if/what they report about the drives.
Interesting. I have several drives that are in various states of ill-health. I monitor them via Acronis Disk Monitor. The stats returned by ADM match those of other SMART tools I use. They are all currently connected to Win7 physical servers. I don't recall ever seeing any pro-active messages from Windows about these drives.
My Win8.1 and Win10 systems are all VMs. I'll have to connect some of these bad drives to the newer systems to see if/what they report about the drives.
@Marc / @Scott:
I certainly hope that Scott doesn't perceive these comments as coming from a bunch of whiners. I am always happy to learn more about the Windows disk subsystem, and I applaud Scott for both his efforts and his openness.
Thanks, Scott! :)
I certainly hope that Scott doesn't perceive these comments as coming from a bunch of whiners. I am always happy to learn more about the Windows disk subsystem, and I applaud Scott for both his efforts and his openness.
Thanks, Scott! :)
Thanks for the detailed discussion - it's long overdue that we hear more about how things really work.
I have workstation with a RAID array of four SSDs that's now almost 3 years old, and the I/O load is heavy every day.
The GUI defrag application in Windows 8.1 shows any form of optimization as "Not Available" (guessing here) because the RAID controller does not support TRIM. The defrag command line will do a traditional defrag if requested, but it's unavailable from the GUI.
In fact, because I understand the value of the file system being kept defragmented, I've done manual defrags via the defrag command line on this array - not every month but about every 6 months - and have not had so much as a single glitch with this SSD array.
And, even without any TRIM support whatsoever, the Garbage Collection logic in the smart controllers of these circa 2011 OCZ Vertex 3 SSDs has kept them running optimally. Performance today is as good as ever.
Somewhere up above I think it was Scott who said that a lot of folks base their decisions on outdated information, and I have to agree with that. Even with the heavy write loads I see, my projected lifespan is way over 10 years. People need not worry so much!
-Noel
I have workstation with a RAID array of four SSDs that's now almost 3 years old, and the I/O load is heavy every day.
The GUI defrag application in Windows 8.1 shows any form of optimization as "Not Available" (guessing here) because the RAID controller does not support TRIM. The defrag command line will do a traditional defrag if requested, but it's unavailable from the GUI.
In fact, because I understand the value of the file system being kept defragmented, I've done manual defrags via the defrag command line on this array - not every month but about every 6 months - and have not had so much as a single glitch with this SSD array.
And, even without any TRIM support whatsoever, the Garbage Collection logic in the smart controllers of these circa 2011 OCZ Vertex 3 SSDs has kept them running optimally. Performance today is as good as ever.
Somewhere up above I think it was Scott who said that a lot of folks base their decisions on outdated information, and I have to agree with that. Even with the heavy write loads I see, my projected lifespan is way over 10 years. People need not worry so much!
-Noel
Some folks reading this might think they having nothing to worry about because they turned off System Restore. Or they might turn-off System Restore to avoid any defrag operations on their SSDs.
It's been my experience running Windows 8.1 that despite having intentionally turned off System Restore, Windows 8.1 automatically re-enabled it. One of the things I always do after doing a clean install is turn off System Restore - I've been doing that since Windows 7 was released.
Over the past month, I've been running three different systems with clean installs of Windows 8.1. On all of them, the OS has automatically turned System Restore back on. I don't know how (but I suspect Windows Update does it) or why, but it is happening.
I have never seen this behavior with Windows 7.
It's been my experience running Windows 8.1 that despite having intentionally turned off System Restore, Windows 8.1 automatically re-enabled it. One of the things I always do after doing a clean install is turn off System Restore - I've been doing that since Windows 7 was released.
Over the past month, I've been running three different systems with clean installs of Windows 8.1. On all of them, the OS has automatically turned System Restore back on. I don't know how (but I suspect Windows Update does it) or why, but it is happening.
I have never seen this behavior with Windows 7.
@Vadim
You did appear very defensive and aggressive in your first comments.
He already admitted he was wrong why do you keep attacking him on that then? If you only want a technical discussion stop the unnecessary remarks.
You are not satisfied easily are you? Scott probably went back and talked it over some more and then comes back with additional information and then you complain he should have supplied it right away.
@Scott
Thanks for supplying the info :).
You did appear very defensive and aggressive in your first comments.
А Microsoft employee publicly states that Windows doesn't defrag SSDs using full stops, then he reiterates his point citing some product team, and now finally states that he was wrong. I guess if Scott looked into the evidence before making his claims, he'd come up with the correct answer sooner.
There's no whining here, except for your attacks. I'm only interested in technical discussion. If Scott doesn't know anything about technical details on this topic, perhaps, he shouldn't be posting false statements on other blogs without checking the facts as well as delivering incomplete info on his blog post called "complete story".
He already admitted he was wrong why do you keep attacking him on that then? If you only want a technical discussion stop the unnecessary remarks.
Scott, thanks for providing additional info you should've supplied in the original post. This just confirms that "once a month" is an approximation. Ok, they just ensure the SSD defrag period is less than 28 days.
You are not satisfied easily are you? Scott probably went back and talked it over some more and then comes back with additional information and then you complain he should have supplied it right away.
@Scott
Thanks for supplying the info :).
So now I'm confused... With Windows 7 and an SSD, should I turn the disk defragmenter service back ON (I disabled it when I first installed the drive 3 yrs ago)? I've turned off System Restore (which had been on). I turned on the defragment service and analyzed the drive (38% fragmented) but didn't run it; my automatic scheduler for defrag is turned off... when I turn it on, the drive doesn't show up anyway). Thanks.
We may need another post on this to distil all the various nuances which have been outlined in the comments. I am taking away no real guidance as is. But the original post was much appreciated, despite the fact that it wound up that noisy little guy.
@dan
You can't defrag SSD on Windows 7 using the built-in Windows defragmenter. Per Microsoft, Windows 8+ is getting defragged with the default settings, and Windows 7 isn't. I'm sure Microsoft can easily clarify this little difference ;)
@Dave
Then you should ask that big quiet guy who posted the real and complete story^^ :)
@Rutger Storm
Let's see. A big and famous Microsoft blogger comes to my blog and starts throwing his weight around claiming I'm incompetent without even reading my post. Then he keeps defending his position, citing some product group. And even when he finally admits he was wrong, he criticizes my analysis in broad terms without linking to it and pointing to particular technical issues.
It's not fair play, but your opinion may be different, of course :)
I'm not attacking him. I'm questioning his technical statements on this topic, because I smell a lack of knowledge depth (expect for the secret knowledge supplied by the product group). This is understandable, but it's not an excuse after a blog entry with such a promising title is posted :)
No, I'm not satisfied easily, because unlike [you and] Scott, I've spent a lot of time researching the issue, so my questions are on target.
If it's a regular defrag, don't call it "intelligent defrag", because you're misleading your audience yet again.
If you mention a special schedule, I want to know where it comes from, i.e. whether it's yet another fantasy of yours or a specific info from the PG. Hence, I don't complain, I just state what I expect from the real and complete story written by a person who has a direct communication line with the PG. Yes, my expectations are high, but I'm not sorry about that.
As a blogger myself, I love loyal readers. So I'm sure Scott appreciates your support :) But I always teach my readers to think critically and don't believe everything they hear or read, even in my blog; I teach them how to check these statements thru experiments. And I never publish a blog post without researching a topic myself, because I don't have a luxury of running to the PG every day ;)
Vadim
You can't defrag SSD on Windows 7 using the built-in Windows defragmenter. Per Microsoft, Windows 8+ is getting defragged with the default settings, and Windows 7 isn't. I'm sure Microsoft can easily clarify this little difference ;)
@Dave
Then you should ask that big quiet guy who posted the real and complete story^^ :)
@Rutger Storm
You did appear very defensive and aggressive in your first comments.
Let's see. A big and famous Microsoft blogger comes to my blog and starts throwing his weight around claiming I'm incompetent without even reading my post. Then he keeps defending his position, citing some product group. And even when he finally admits he was wrong, he criticizes my analysis in broad terms without linking to it and pointing to particular technical issues.
It's not fair play, but your opinion may be different, of course :)
He already admitted he was wrong why do you keep attacking him on that then? If you only want a technical discussion stop the unnecessary remarks.
I'm not attacking him. I'm questioning his technical statements on this topic, because I smell a lack of knowledge depth (expect for the secret knowledge supplied by the product group). This is understandable, but it's not an excuse after a blog entry with such a promising title is posted :)
You are not satisfied easily are you? Scott probably went back and talked it over some more and then comes back with additional information and then you complain he should have supplied it right away.
No, I'm not satisfied easily, because unlike [you and] Scott, I've spent a lot of time researching the issue, so my questions are on target.
If it's a regular defrag, don't call it "intelligent defrag", because you're misleading your audience yet again.
If you mention a special schedule, I want to know where it comes from, i.e. whether it's yet another fantasy of yours or a specific info from the PG. Hence, I don't complain, I just state what I expect from the real and complete story written by a person who has a direct communication line with the PG. Yes, my expectations are high, but I'm not sorry about that.
As a blogger myself, I love loyal readers. So I'm sure Scott appreciates your support :) But I always teach my readers to think critically and don't believe everything they hear or read, even in my blog; I teach them how to check these statements thru experiments. And I never publish a blog post without researching a topic myself, because I don't have a luxury of running to the PG every day ;)
Vadim
Ahhh Hanselman - doing the hard work and taking the flak, so I don't have to! Thanks Scott!
Hi Scott and Vadim - as the specialists of defrag
It's pretty good to get this information about defragmentation of SSDs.
But there is another problem left on server SKUs and defragmentation of standard SSDs.
Within the documentation it is told that
- defragmentation will start if the system is idle, and
- defragmentation will stop if the systems is leaving idle mode.
This makes sense fort workstation operating systems, but it is problem on servers.
Based on this we found:
defragmentation will start as scheduled, but it will never end on server SKUs and disk performance will degrade while defragmentation is running.
Problem is that the machine is leaving idle mode , and therefore interrupting defragmentation, if someone generates mouse or keyboard interaction (interrupts) on the console connected devices.
BUT: normally no one will work interactively on a server SKU.
Maybe you can put some light on this issue.
regards
andy
It's pretty good to get this information about defragmentation of SSDs.
But there is another problem left on server SKUs and defragmentation of standard SSDs.
Within the documentation it is told that
- defragmentation will start if the system is idle, and
- defragmentation will stop if the systems is leaving idle mode.
This makes sense fort workstation operating systems, but it is problem on servers.
Based on this we found:
defragmentation will start as scheduled, but it will never end on server SKUs and disk performance will degrade while defragmentation is running.
Problem is that the machine is leaving idle mode , and therefore interrupting defragmentation, if someone generates mouse or keyboard interaction (interrupts) on the console connected devices.
BUT: normally no one will work interactively on a server SKU.
Maybe you can put some light on this issue.
regards
andy
As I am also interested in this topic, let me share my own feelings here.
First of all, I do really understand and second what Vadim expects from the vendor:
MS should issue a properly(!) written KB article about the detailed behavior of defragmenting and SSDs under W8.x/WS2012(R2)
But even better (as the quality of KB articles declined since the introduction of the "improves the reliability, stability, and performance of <INSERT ANY MS PRODUCT NAME HERE>" type of changelogs), it would be more appropriate (and more satisfy the expectations of all people who contributed to this thread) if the storage team blog would accept the challenge, and write a great article about this topic.
Actually I have no idea, who supposed to be "THE storage team" inside that company:
- http://blogs.technet.com/b/filecab/
- http://blogs.technet.com/b/askcore/archive/tags/storage+and+file+systems/default.aspx
- or http://blogs.technet.com/b/josebda/
I think I also understand what Jim Burd misses in this post: some hard facts that cant be argued with. I cannot explain why, but reading through this story and the partial snippets Scott glued together from that secret product group replies makes me feel there is some level of BS in this topic. I recommend everyone to read these couple of articles below, and you will understand what I mean:
http://blogs.msdn.com/b/ntdebugging/archive/2008/07/03/ntfs-misreports-free-space.aspx
http://blogs.msdn.com/b/ntdebugging/archive/2008/10/31/ntfs-misreporting-free-space-part-2.aspx
http://blogs.technet.com/b/askcore/archive/2009/10/16/the-four-stages-of-ntfs-file-growth.aspx
I think you now understand whats the difference between a loosely worded reply comment and a professionally written article about the same topic. Hardly anybody can complain about being non-technical in any of the above referenced blogposts.
If Scott would get a reply like the above one from the PG, it would satisfy the curiosity of all the people here, and we would have already closed the argument here and could go on to other issues in the IT world.
I feel the pain of others, who think were neglected by MS when their question on answers.microsoft portal remain unanswered for years. I dont want to hijack this thread and wont post the URL to my question but I also have several topics left without any feedback (for example about the inability of WP8 to inform me what policies the company is enforcing on my phone when I want to establish an EAS session, but thats an offtopic thing).
At the end of my post, so just to be constructive, here is the answer to the shady topic of overfragmented files:
How fragmentation on incorrectly formatted NTFS volumes affects Exchange
http://bit.ly/1HZvoSx
NTFS does have it’s limitations with the overall size of this attribute list per file and can have roughly around 1.5 million fragments. This is not an absolute maximum, but is around the area when problems can occur.
So at least this unknown part of the discussion is clarified at least :)
First of all, I do really understand and second what Vadim expects from the vendor:
MS should issue a properly(!) written KB article about the detailed behavior of defragmenting and SSDs under W8.x/WS2012(R2)
But even better (as the quality of KB articles declined since the introduction of the "improves the reliability, stability, and performance of <INSERT ANY MS PRODUCT NAME HERE>" type of changelogs), it would be more appropriate (and more satisfy the expectations of all people who contributed to this thread) if the storage team blog would accept the challenge, and write a great article about this topic.
Actually I have no idea, who supposed to be "THE storage team" inside that company:
- http://blogs.technet.com/b/filecab/
- http://blogs.technet.com/b/askcore/archive/tags/storage+and+file+systems/default.aspx
- or http://blogs.technet.com/b/josebda/
I think I also understand what Jim Burd misses in this post: some hard facts that cant be argued with. I cannot explain why, but reading through this story and the partial snippets Scott glued together from that secret product group replies makes me feel there is some level of BS in this topic. I recommend everyone to read these couple of articles below, and you will understand what I mean:
http://blogs.msdn.com/b/ntdebugging/archive/2008/07/03/ntfs-misreports-free-space.aspx
http://blogs.msdn.com/b/ntdebugging/archive/2008/10/31/ntfs-misreporting-free-space-part-2.aspx
http://blogs.technet.com/b/askcore/archive/2009/10/16/the-four-stages-of-ntfs-file-growth.aspx
I think you now understand whats the difference between a loosely worded reply comment and a professionally written article about the same topic. Hardly anybody can complain about being non-technical in any of the above referenced blogposts.
If Scott would get a reply like the above one from the PG, it would satisfy the curiosity of all the people here, and we would have already closed the argument here and could go on to other issues in the IT world.
I feel the pain of others, who think were neglected by MS when their question on answers.microsoft portal remain unanswered for years. I dont want to hijack this thread and wont post the URL to my question but I also have several topics left without any feedback (for example about the inability of WP8 to inform me what policies the company is enforcing on my phone when I want to establish an EAS session, but thats an offtopic thing).
At the end of my post, so just to be constructive, here is the answer to the shady topic of overfragmented files:
How fragmentation on incorrectly formatted NTFS volumes affects Exchange
http://bit.ly/1HZvoSx
NTFS does have it’s limitations with the overall size of this attribute list per file and can have roughly around 1.5 million fragments. This is not an absolute maximum, but is around the area when problems can occur.
So at least this unknown part of the discussion is clarified at least :)
@Vadim Sterkin
You are coming off very poorly in these comments. It's not Scott's job to know anything about SSDs, the Windows storage sub-system, or to answer you.
This is his PERSONAL blog, which he typically talks about the technological things that he enjoys. He tried to get more information from others that he works with, however that does not make it his job to write and release documentation.
From what I've seen, you assumed there was a bug in Windows because Windows 7 documentation states that Windows will never defrag an SSD. Scott (armed with the incorrect info) corrected you, and you did not like it.
So now that you know he was wrong, not because you proved it, but because he dug deeper and figured it out, you have something to hang over his head. Even though you were also wrong in stating that it was a bug. And to say that it still could be a bug without seeing internal documentation is ridiculous.
There is no technical discussion to be had, the optimization routine in Windows 8 that defragments the disk is by design. And is necessary for an SSD, as an OS drive, to continuously work long term due to the fragmentation limits expressed by Scott.
I think most people would agree that they never heard of fragmentation limits, however most would agree that it sounds likely, especially on lower cost SSDs.
They have to compensate for low memory devices, or poorly engineered devices all the time, why would this be any different?
It's almost like you can't believe you were wrong, and that they wouldn't learn anything from previous operating systems that supported SSDs.
You are coming off very poorly in these comments. It's not Scott's job to know anything about SSDs, the Windows storage sub-system, or to answer you.
This is his PERSONAL blog, which he typically talks about the technological things that he enjoys. He tried to get more information from others that he works with, however that does not make it his job to write and release documentation.
From what I've seen, you assumed there was a bug in Windows because Windows 7 documentation states that Windows will never defrag an SSD. Scott (armed with the incorrect info) corrected you, and you did not like it.
So now that you know he was wrong, not because you proved it, but because he dug deeper and figured it out, you have something to hang over his head. Even though you were also wrong in stating that it was a bug. And to say that it still could be a bug without seeing internal documentation is ridiculous.
There is no technical discussion to be had, the optimization routine in Windows 8 that defragments the disk is by design. And is necessary for an SSD, as an OS drive, to continuously work long term due to the fragmentation limits expressed by Scott.
I think most people would agree that they never heard of fragmentation limits, however most would agree that it sounds likely, especially on lower cost SSDs.
They have to compensate for low memory devices, or poorly engineered devices all the time, why would this be any different?
It's almost like you can't believe you were wrong, and that they wouldn't learn anything from previous operating systems that supported SSDs.
@Brian:
why is it so difficult to understand that some people dont swallow made-up answers and want to find out the real truth? Some people are fine with cheap/lower quality solutions, while some pursue the perfectness.
why is it so difficult to understand that some people dont swallow made-up answers and want to find out the real truth? Some people are fine with cheap/lower quality solutions, while some pursue the perfectness.
Scott, how about SSDs in USB 3 enclosures? I'm running Windows 8.1 and the Optimize Drives feature has a Media Type of Hard Disk Drive instead of Solid State Drive for my Crucial MX100 512GB SSD. I'm assuming this means TRIM isn't enabled and it is being defragged like a traditional HDD. I know it's not your question to answer but this is the best discussion of TRIM and Windows I've found. The question seems to be asked quite a bit with nobody really knowing. Would you mind querying your Windows Storage Developers once more?
Thanks!
Thanks!
Hey Scott, I don't want to bust you up for trying your best to get accurate info. I wanted to share this though. http://i59.tinypic.com/2ufczf4.png I know you didn't specify brands and maybe what it's saying there is true for Intel-only.
But I think we're left with more questions than answers. Maybe we could get MS and Intel to come together and issue a joint statement and article on this topic?
But I think we're left with more questions than answers. Maybe we could get MS and Intel to come together and issue a joint statement and article on this topic?
Ok , it seems now the owner of this blog does not want to bother commenting on this unclosed topic anymore. I still feel that the semi-official answer was a made-up one, and did not help to clear the confusion at all, I personally am as uncomfortable whats the truth ad I was before reading this blog.
When I read a blog post of Mark Russinovich about LITERALLY ANYTHING he writes, at the bottom of the page I end the reading as if that topic has been fully explained, and I dont question the knowledge of the author in that topic. Here, I totally felt the opposite. Anybody commenting above "felt satisfied" is not at the same level as me or Vadim, and can be easily mislead by semi-pro madeup mumbo-jumbo speech.
When I read a blog post of Mark Russinovich about LITERALLY ANYTHING he writes, at the bottom of the page I end the reading as if that topic has been fully explained, and I dont question the knowledge of the author in that topic. Here, I totally felt the opposite. Anybody commenting above "felt satisfied" is not at the same level as me or Vadim, and can be easily mislead by semi-pro madeup mumbo-jumbo speech.
Vadim Sterkin says...
"You can't defrag SSD on Windows 7 using the built-in Windows defragmenter"
Sorry this is wrong
I have windows 7 64 and a Samsung 840 EVO 250GB
the IOPS dropped by over 8x overnight
so did the read writes
I had over 120GB of free space so the slow down was not a full SSD
I was going to use the Samsung wiper and reinstall windows to bring back the speed but just for a test I opened the win 7 defrag tool, clicked the c: SSD, closed my eyes and ran it
it analysed and then defragged the SSD
I then run the Samsung benchmarking tool and all was back to as good as new.
I don't think this was because the drive was defragged but more to do with the data being refreshed
trim was always on and I forced it to trim in the tools
the outcome shocked me as I thought we could not defrag a ssd and that it would not help
"You can't defrag SSD on Windows 7 using the built-in Windows defragmenter"
Sorry this is wrong
I have windows 7 64 and a Samsung 840 EVO 250GB
the IOPS dropped by over 8x overnight
so did the read writes
I had over 120GB of free space so the slow down was not a full SSD
I was going to use the Samsung wiper and reinstall windows to bring back the speed but just for a test I opened the win 7 defrag tool, clicked the c: SSD, closed my eyes and ran it
it analysed and then defragged the SSD
I then run the Samsung benchmarking tool and all was back to as good as new.
I don't think this was because the drive was defragged but more to do with the data being refreshed
trim was always on and I forced it to trim in the tools
the outcome shocked me as I thought we could not defrag a ssd and that it would not help
@Brian
You got it wrong. Scott didn't dig deeper of figure anything out. He just asked the product group and published their answer, for which I thanked him immediately. He also translated the answer from dev English to human English, which most blog readers need desperately.
Indeed, there's no technical discussion to be had with people not familiar with the issue or/and who happily accept everything MSFT employees tell them (and the more full stops they use, the better :)
@Richard
It's all about the right expectations. This is not Scott's area of expertise. He could've been just the messenger, but decided to go beyond that with a viral blog post title and his own interpretations of the dev answer.
@PC
My statement "You can't defrag SSD on Windows 7 using the built-in Windows defragmenter" does not apply to situations when:
• SSD is treated as HDD for whatever reason
• people think defrag has occured while it actually has not
Make sure you've got the latest firmware (your drive needs it badly) and all Windows updates. Then run winsat and repeat defrag. Then you can drop your event 258 logs (or better CheckBootSpeed results) on PasteBin, and I'll take a look. Ping me on Twitter.
You got it wrong. Scott didn't dig deeper of figure anything out. He just asked the product group and published their answer, for which I thanked him immediately. He also translated the answer from dev English to human English, which most blog readers need desperately.
Indeed, there's no technical discussion to be had with people not familiar with the issue or/and who happily accept everything MSFT employees tell them (and the more full stops they use, the better :)
@Richard
It's all about the right expectations. This is not Scott's area of expertise. He could've been just the messenger, but decided to go beyond that with a viral blog post title and his own interpretations of the dev answer.
@PC
My statement "You can't defrag SSD on Windows 7 using the built-in Windows defragmenter" does not apply to situations when:
• SSD is treated as HDD for whatever reason
• people think defrag has occured while it actually has not
Make sure you've got the latest firmware (your drive needs it badly) and all Windows updates. Then run winsat and repeat defrag. Then you can drop your event 258 logs (or better CheckBootSpeed results) on PasteBin, and I'll take a look. Ping me on Twitter.
I am building a new system with ssd for the first time. I have been digging around for information. I understand for the most part that defrag is bad, fragmentation still happens on ssd. What I have not seen is if win 8 does defrag once a month how much damage is it really doing to my ssd? Also, couldnt I modify the schedule? Increase it to 2 months if my drive isnt overly used?
Thanks for any help,
Todd
Thanks for any help,
Todd
Comments are closed.