back to article Peeking up the skirt of Microsoft's hardy ReFS

As reported this past fortnight, Microsoft's new Storage Spaces for Windows 8 is only half the story; the operating system builder is also throwing in a new Resilient File System (ReFS) while retaining most NTFS features and semantics. Storage Spaces is a Windows 8 feature that enables a PC user to aggregate physical disk …

COMMENTS

This topic is closed for new posts.
  1. Joe Montana
    FAIL

    More lock-in

    I guess FUSE-NTFS was getting too good, so now they need to migrate to another proprietary filesystem in order to continue their lock-in scheme...

    1. John Riddoch

      I think you're being a bit melodramatic about this. The only filesystem that has those kind of features is ZFS as far as I'm aware and it probably doesn't support the kind of things Windows requires (and is under a license which MS probably don't want to sign up to). I doubt their prime concern here was killing open source compatibility, although I suspect it came up. I also don't see anything here which precludes you continuing to use vanilla NTFS.

      1. Oninoshiko
        Boffin

        actually,

        CDDL code would be vary easy for them to incorporate into windows. The CDDL is only scoped to the actually covered files (unlike the GPL is it non-viral). Therefore they would not have to worry about making any changes to how windows is licensed, they would just have to provide the code to their version of the specific CDDL licensed files they used under the terms of the CDDL, not the entire OS (nor even the other original files created as part of the porting process).

        I'm curious, what things do you think windows needs that ZFS does not do?

  2. The BigYin

    So...

    ...how does it stack up against the likes of BTRFS (also B-tree) or Ext4 or any of the other options available?

    This just smacks of "N.I.H." syndrome and, considering it's MS, "Using an open system would increase competition".

    1. Ru
      Headmaster

      More 'live by the sword...' than NIH

      You can't go around threatening all and sundry with software patent infringement unless you're absolutely certain that your own stuff is unimpeachable. MS have come unstuck here in the past as I'm sure you're aware, but ultimately the safest way to write software in the sort of Mutually Assured Destruction business environment in which MS likes to operate is to write everything yourself from the ground up, using designs you've developed yourself.

      They've basically painted themselves into a corner, and that's why they product so much "me, too!" stuff instead of using liberally licensed open source software.

  3. William Boyle
    Thumb Down

    Other OS support?

    So, how much information will MS provide to help us create Linux drivers for this file system? Or will we have to reverse-engineer it from scratch? I kind of suspect they will make it as difficult as possible, even to the extent of claiming copyright and/or patent privileges...

  4. Anonymous Coward
    Anonymous Coward

    As long as I can delete my data and be confident I have actually deleted it I will be happy.

    Otherwise I suspect a flag will be set in metaddata showing the file as deleted so it cant be seen.

  5. a_mu

    another file system

    why another file system ?

    whats wrong with the other ones out there

    apart from they are not Microsoft locked.

    how can companies and governments insist on open file structures, but not open file systems ?

    if I plug a disc into a different system I'd expect , with the appropriate pass words etc, to be able to read it,

    Ha

    1. Ian McNee
      Linux

      Because Micros~1 doesn't want to slip more than ten years behind Linux?

      The kind of features they are including with their "new" file systems put Windoze on a par with Linux in the mid-noughties as opposed to NTFS's equivalence to the mid-nineties.

      Aggregated storage? Jeez! Amazing! I forget how long ago I first toyed with LVM on Linux!

      So we may have to use our Windoze VM for a while to retrieve data from borked ReFS storage while waiting for stable kernel drivers. No biggie.

      1. Anonymous Coward
        Anonymous Coward

        Aggregated storage is not what you think it is

        Windows can do aggregated storage in the Linux sense: that is, mount a filesystem somewhere in an existing filesystem. But this isn't terribly helpful if you need extra space somewhere else in the filesystem hierarchy.

        What people actually want to do is have a single filesystem transparently spanning multiple drives, preferably with the ability to add and remove drives without rebuilding the filesystem. ZFS can do this now. Btrfs implements this on Linux, but it is in no sense production ready. ReFS will add this for Windows.

        1. Ian McNee
          Flame

          LVM is not what *YOU* think it is - it *IS* aggregated storage

          Try to engage your brain before you call someone else stupid. Yes we *ALL* know that mounting disk partitions in a filesystem is not aggregated storage.

          Look up LVM - Logical Volume Manager. If you don't know how to use Google here's a link: http://www.centos.org/docs/5/html/Cluster_Logical_Volume_Manager/

          LVM *IS* aggregated storage, it *IS* used in enterprise storage solutions, it has been around in Linux since the late nineties and has been a mature product for 5+ years.

          1. Vic

            > LVM *IS* aggregated storage

            ... *can be* aggregated storage. can be many things, in fact.

            It's quite wonderful to watch people's faces the first time they see an lvresize / resize2fs play...

            Vic.

          2. Anonymous Coward
            Anonymous Coward

            @Ian McNee

            Tee hee - you're twelve years old, you? All-capital words and asterisks too! The Reg should make a stamping foot icon for you ...

        2. Vic

          > What people actually want to do is have a single filesystem transparently

          > spanning multiple drives

          Like LVM, then.

          Vic.

    2. Daniel 4

      Key features are NOT in common products

      Actually, as nice as the easy management features sound, the real strengths here are features such as checksums for detecting silent data errors, copy-on-write, etc.

      -d

  6. DrXym

    The pertinent point

    Is this is for Windows Server edition. It won't be in bog standard Windows Paeon edition. It also doesn't support booting from ReFS. Nor can you convert a disk from NTFS to ReFS despite them sharing some similarity.

    This to me suggests Microsoft is being cautious. Server edition gets it first, but it's unlikely to show up anywhere else until Windows 9 or some major service pack.

    1. dogged

      It's also interesting in terms of SKUs

      The blog explicitly references "Windows 8 Server" which automatically deprecates the "Server 20xx" naming convention (for the better, I think) and leads one to wonder whether MS are attempting a simultaneous Server/Desktop release.

      They haven't done that since Win2K.

      1. Anonymous Coward
        Anonymous Coward

        @dogged

        Windows 7 and Windows Server 2008 R2 simultaneous client & server SKU releases, built from the same source branch (not just staggered by 1 service pack like Longhorn).

  7. Phil Bennett
    Thumb Up

    Checksumming!

    Microsoft have (finally) seen the advantages of ZFS, and implemented (a few) of the the features. There seems to be a fair bit of conflicting info floating around on the web, but if ReFS with Storage Spaces does have full checksumming then I might consider it as an upgrade path for my Solaris / ZFS file server.

    Yes, I agree, it'd be nice if they'd play nicely with open source and help get BTRFS up to production standard (or somehow make a deal with Oracle and use ZFS) but failing that this is a good second best.

    1. Anonymous Coward
      Anonymous Coward

      Err...

      To be fair, ZFS was hardly in wide scale use in 2008 when the last NT server was released.

  8. Earle M
    Unhappy

    There are many NTFS features that will not be present on ReFS

    There are many people that are not happy about the loss of named streams, hard-links, and compression.

    From the Q&A at the end of the 'Building Windows 8' blog entry:

    Q) What semantics or features of NTFS are no longer supported on ReFS?

    The NTFS features we have chosen to not support in ReFS are: named streams, object IDs, short names, compression, file level encryption (EFS), user data transactions, sparse, hard-links, extended attributes, and quotas.

    1. Robert Carnegie Silver badge

      Hang on,

      doesn't Windows itself depend on some of those features? Hard links for the WinSXS folder, or am I mistaken?

      1. Anonymous Coward
        Anonymous Coward

        Win SXS

        It actually uses reparse points which are links but for linking directories. From Vista onwards there are file and directory links, it appears that it's the file links that are being removed. It's a shame that they're dropping this as POSIX is a nice to have, I quite like having SFU installed on a server and this would seem to suggest that the new filesystem can't be used with SFU.

        If you're interested a common WinSXS reparse point you could look at is:

        fsutil reparsepoint query c:\windows\assembly\gac_32\system.enterpriseservices\2.0.0.0__b03f5f7f11d50a3a

    2. mark jacobs
      Childcatcher

      We use compression extensively on our Windows servers. What is going to happen to us? They just don't care...

      1. Anonymous Coward
        Anonymous Coward

        @Mark jacobs

        They're not making you use ReFS, NTFS is still supported, so it looks like they do in fact care.

  9. Bruce Ordway

    WinFS

    Whatever happened to the file system as database?

    1. The BigYin

      It's here

      ZFS and BTRFS to name but two (with varying degrees of implementation).

      Of course, Windows can't use them.

  10. Anonymous Coward
    Anonymous Coward

    So does this mean I won't have to defrag my file system any more?

    Seriously; what kind of shitty server OS requires me to do that *cough* Windows Server 2008 *cough*? Unbelievable.

  11. Eduard Coli
    Facepalm

    Ugg

    Please M$ do us all a favor and stop with the half assed, brain dead copies of other peoples file systems.

    You know it won't work, not at least until version 3 with version 5 being the one that everyone expected in version 1.

    All you have to do is integrate ZFS and even the license still lets you stay kind of closed source.

    Copy Apple if you must.

    1. Anonymous Coward
      Anonymous Coward

      Seriously?

      Is that the best comment you've got?

      MS implement functionality that's available elsewhere and it's brain-dead copying, let me guess, when Linux vendors do it (let's take journaling file systems as an example) it'd be implementing best practice?

  12. Frederic Bloggs
    Meh

    Allocate-on-write

    Allocate-on-write and (although it doesn't say so) ordered metadata update? Are they going to (try to) patent it?

    George 3 1965.

  13. Anonymous Coward
    Anonymous Coward

    File systems are proper hard

    Wonder how long its been in development? The blog says they've given it loads of thrashing with their test cases but that is a far cry from the real world.

    No matter how hard you play in the lab, users will find another "corner case" that you missed.

    My favourite example of this is an image (say from dd) of a ReiserFS3 lies on a RFS. One day you run fsck (chkdsk for the uninitiated) and find the image and the FS intertwined because the checker had a hard time telling the difference between them. Don't know if this feature is confined to RFS either.

    Read the online discussions from FS developers and you'll wonder why you use anything that isn't at least five years old in the real world ...

    Finally, no matter how fancy it is, you'd better back it up properly and give it a UPS and a good hardware RAID or SAN if you care about your data.

    Cheers

    Jon

    1. Kebabbert

      @gerdesj

      "...Finally, no matter how fancy it is, you'd better back it up properly and give it a UPS and a good hardware RAID or SAN if you care about your data...."

      Backups are good, but how do you know that the backup does not contain corrupted data? I have heard of old backupped files, and long time after, the files are not readable. The zip file is not extractable, the media mp3 is not playable, etc.

      A backup needs checksums to detect and protect against these data corruption errors. That is the reason MS is adding checksums for metadata. However, that does not suffice, the data need also to be checksummed. Thus, ReFS is not that safe. As research shows, NTFS is not safe either. Nor is XFS, ReiserFS, JFS, etc which researchers prove. There is a reason ECC RAM uses checksums, so should filesystems.

      Thus, a backup on a hardware raid or SAN does not suffice, and you might still have corrupted data. There are many texts on this, I hope you have not missed them.

  14. steeplejack
    Devil

    NTFS certainly does lose data.

    Hope the new error-detection highlights the cases where NTFS loses the entire tail-end of a directory (and all the respective files have disappeared).

  15. John Halewood

    Rather confused by the fact that this will come out for the server version of the OS, but is missing features (that they have "chosen" not to include) such as file level encryption and quotas which are exactly the sort of things it would seem to me that you're likely to want in a server rather than a desktop. Maybe they're opening up the market for a bunch of third party products to provide the missing bits?

    1. Anonymous Coward
      Anonymous Coward

      Err

      Surely you want encryption on the desktop side, servers should be nicely secured both physically and logically, anyone can get their nite on a laptop/desktop.

  16. Steve Davies 3 Silver badge
    Linux

    I wonder how many patents MS has on this lot

    They certainly won't want those pesky freetards reverse engineering it now will they?

    1. Charles Manning

      Well it is certainly greater than the number of valid patents they hold.

  17. kissingthecarpet
    Linux

    Surely the EU-mandated

    interoperability requirement means that MS have to allow interaction with ReFS, & thereby reveal enough information for open-source drivers to be written? Well, let's hope it does anyway.....

    1. Anonymous Coward
      Anonymous Coward

      Err?

      Interoperability would be the ability to read and write files with no extra hooks into the filesystem available to MS than are available to everyone else. It would not be assisting rival companies to develop your technology for their competing platform.

  18. amanfromMars 1 Silver badge

    SMART Virtual Machine Self Actualisation ..... Remote EnigmatICQ Virtualisation

    " and data is verified and auto-corrected using checksums" …

    Ah, yes, those Magic Semantic Algorithms in Service of Universal Singularity …… which suggest that do indeed Exist and are in Present Future Virtual Play, Microsoft/Surendra Verma/Azure.

    A lady on the radio has just said that she is passionate and not hysterical, and indeed she was and talked perfect sense.

  19. Robert Heffernan
    Mushroom

    Hmm

    Before I make my comment, I would like to point out that by no means am I a Microsoft fanboy (I have a server running Citrix (Linux) splitting it up into virtual servers running a mix of Windows or Linux depending on the workload being served) or trying to defend Microsoft in any way but...

    If Microsoft said "Ok, so this new FS we are after needs to basically cover all the functions currently offered by the Linux LVM system, so for the sake of ease of implementation and compatibility lets just implement LVM" then the Open Source zealots would haul Microsoft up in front of the first court that would take their case demanding that the Windows code base be opened up to everyone.

    I understand the need for open and available specs and drivers for filesystems, but for Microsoft to even think about risking implementing an Open Source filesystem in Windows then they would be extremely negligent to their shareholders and just bad business sense. The only logical and legal choice they have is to implement a new custom closed source filesystem.

    While it doesn't do a lot of things NTFS can do and does a lot of things NTFS can't, it still sounds like a pretty decent filesystem, While the whole FAT/FAT32 thing was a bit of a debacle, being an old legacy filesystem, NTFS was actually pretty good, so I am looking forward to giving ReFS a try to see how it performs. I also think the EXT filesystem is pretty good and I would love a spare box to give ZFS a try!

    *Nuke: Closest thing referencing the bunker i'll need to deal with all the inevitable flaming I am about to get!

    1. Vic

      > the Open Source zealots would haul Microsoft up in front of the first

      > court that would take their case

      This is untrue.

      If Microsoft were to implement LVM, the entire Open Source community would say "excellent - well done".

      If Microsoft were, instead, just to take the LVM source code and incorporate it into Windows, the copyright owners (primarily Red Hat) would insist on the source being open according the the GPLv2 licence under which it is distributed.

      This is the principal difference between open- and closed-source; open-sourcers *want* stuff to proliferate. The GPL, for example, gives you explicit rights to study the code with no restriction on the purpose of that study; you are *encouraged* to find out how it works and then write your own version.

      All that is really frowned upon is blatant copyright infringement. MS got caught out that way once before, and were obliged to release some of their code under GPL. I can't see them doing that again for many years...

      Vic.

  20. Leedos
    Thumb Up

    Client side too please.

    Now just put this in Windows 8 Professional and make 85% of El Reg readers very happy. The other 15% can continue to whine here and not benefit from it.

This topic is closed for new posts.

Other stories you might like