Welcome to the MacNN Forums.

If this is your first visit, be sure to check out the FAQ by clicking the link above. You may have to register before you can post: click the register link above to proceed. To start viewing messages, select the forum that you want to visit from the selection below.

You are here: MacNN Forums > Software - Troubleshooting and Discussion > macOS > Journaling File System

Journaling File System
Thread Tools
SJ
Junior Member
Join Date: Dec 2000
Location: Earth
Status: Offline
Reply With Quote
Jan 16, 2001, 05:39 AM
 
One of the things I seem to be hearing arround the place these days is about Journaling File Systems. The Be file system has been said to be wonderful.

My Questions are;

1). Are they are great as I have heard?
2). Should HFS+ be replaced with a JFS?
3). What other things should the FS in OSX have to make people go "Wow". (FS Level encryption etc....)
4). Are JFS's any good for things like video editing?

Cheers
     
not important
Guest
Status:
Reply With Quote
Jan 17, 2001, 08:48 PM
 
From what I have heard and seen, journaling file systems don't get corrupted. For example, you could just turn your computer off at any time and the only thing you would lose would be the data that is waiting in the buffer. I think that it also makes the hard drive integrity checks at startup unnecessary, which would lead to faster boot times. I have used this in Linux and really didn't notice any difference in the handling of applications, but then again I didn't use it much (it wasn't my computer).
I'm pretty sure that the OS has to support it, though. I doubt OS 9/X does.
Have you heard of a database FS? I hear work is going on for something like this. You can access your database as if it were a hard drive. Sounds pretty cool if databases are something you work with alot.
     
me
Mac Enthusiast
Join Date: Jun 2000
Location: Seattle, WA
Status: Offline
Reply With Quote
Jan 17, 2001, 10:05 PM
 
The wonderful thing about Mac OS X is abstraction is used as much as possible. When OS X wants to read or write a file it calls on a Virtual File System which is an abstraction between the OS and the actual file system(s). The VFS uses a plug-in architecture to access actual file systems. Basically that means if you write a plug-in which turns the VFS messages into the FS specific functions...you can have any file system you want. I'm not sure if this is completely true for boot partitions, but certainly for any other partition.

This is different from other OSes like the Classic Mac OS or Linux in which the file system access is hardcoded into the kernel.

In short - write plug-in, get access to any FS you want.
     
tz3 2k1
Guest
Status:
Reply With Quote
Jan 17, 2001, 10:12 PM
 
Originally posted by me:
The wonderful thing about Mac OS X is abstraction is used as much as possible. When OS X wants to read or write a file it calls on a Virtual File System which is an abstraction between the OS and the actual file system(s). The VFS uses a plug-in architecture to access actual file systems. Basically that means if you write a plug-in which turns the VFS messages into the FS specific functions...you can have any file system you want. I'm not sure if this is completely true for boot partitions, but certainly for any other partition.

This is different from other OSes like the Classic Mac OS or Linux in which the file system access is hardcoded into the kernel.

In short - write plug-in, get access to any FS you want.
okay this is all good, but how does the FS for OSX match up to the one in Beos or the JFS? what are our advantages/disadvantages, what are their advantages/disadvantages...on another note, does anyone know if multiple/virtual desktops is to be a part of OSX(without a third party add-on)?
     
Scott_H
Professional Poster
Join Date: Jan 2000
Status: Offline
Reply With Quote
Jan 17, 2001, 11:43 PM
 
OS X will install on the HFS+ unless you format for something different. I would expect XFS to come at some point in time. I think that's journaling AFAIK.
     
tz3gm 2k1
Guest
Status:
Reply With Quote
Jan 18, 2001, 12:20 AM
 
Originally posted by Scott_H:
OS X will install on the HFS+ unless you format for something different. I would expect XFS to come at some point in time. I think that's journaling AFAIK.
XFS is better than JFS? do you know where there would be some online documentation i can look at? anyway, does apple have someting that says that they will use XFS in the future...so, XFS is superior to JFS? in instances where the computer crashes and all those other good stuff...thanks again
     
Scott_H
Professional Poster
Join Date: Jan 2000
Status: Offline
Reply With Quote
Jan 18, 2001, 01:29 AM
 
I don't know it's not my area.
     
hmurchison2001
Senior User
Join Date: Jan 2001
Location: Seattle
Status: Offline
Reply With Quote
Jan 18, 2001, 02:56 AM
 
Originally posted by tz3gm 2k1:
XFS is better than JFS? do you know where there would be some online documentation i can look at? anyway, does apple have someting that says that they will use XFS in the future...so, XFS is superior to JFS? in instances where the computer crashes and all those other good stuff...thanks again
http://www.sgi.com/software/xfs/overview.html

XFS is opensource so Apple could use it at any time. While I don't know what the limitations of Journaling FS I do beleive they exist and therefore have become standaridize in most platforms.
http://hmurchison.blogspot.com/ highly opinionated ramblings free of charge :)
     
eno
Banned
Join Date: Sep 2000
Location: Fightclub
Status: Offline
Reply With Quote
Jan 18, 2001, 02:59 AM
 
XFS is the best FS in the world:
http://www.linuxgazette.com/issue55/florido.html

I wish the people in this forum knew a bit more. Perhaps hang around slashdot and get educated. The slashdot denizens might be opinionated and pigheaded, but at least they know something.
     
Spheric Harlot
Clinically Insane
Join Date: Nov 1999
Location: 888500128, C3, 2nd soft.
Status: Offline
Reply With Quote
Jan 18, 2001, 08:53 AM
 
Originally posted by eno:
I wish the people in this forum knew a bit more. Perhaps hang around slashdot and get educated. The slashdot denizens might be opinionated and pigheaded, but at least they know something.
Well, isn't that why we have you, ya arrogant sop!?

Love,

-chris.
     
mitchell_pgh
Posting Junkie
Join Date: Feb 2000
Location: Washington, DC
Status: Offline
Reply With Quote
Jan 18, 2001, 10:11 AM
 
Then again if you posted this on /. they probably wouldn't even care about the topic or if they did mention it, you would have 120 responses before you even say a new topic... They are a bunch of freaks hitting refresh every five seconds...
     
RichardS
Forum Regular
Join Date: Jan 2000
Location: California
Status: Offline
Reply With Quote
Jan 18, 2001, 11:11 AM
 
Apple cannot switch to any new File System until resource forks die.

That means Classic must no longer exist before we can dump HFS+
     
Millennium
Clinically Insane
Join Date: Nov 1999
Status: Offline
Reply With Quote
Jan 18, 2001, 11:26 AM
 
Journaling filesystems are slightly slower than non-journaling ones, because of the overhead of keeping the journal. However, this slight performance hit is far outweighed by the benefits. Journaling filesystems are, to say the least, Very Cool Things.

This said, if OSX were to incorporate a JFS it would have to be something compatible with Classic. This is, of course, no easy task, particularly if OS9 won't be updated past 9.1.

Now, there are several very good JFS systems out there. SGI's XFS is a popular choice. Linux has at least three right now: ReiserFS (the current favorite, and the first to actually make it into the kernel), XFS, and ext3 (an extension to the current default ext2 system that adds journaling; this one has the advantage of being backward-compatible).

Theoretically, it should be possible to add journaling to HFS+ (HFS-J, perhaps?) in a similar manner to how it was added to ext2 to make ext3. However, in practice that's not nearly so easy, particularly if it were to be accessed by both Classic (which couldn't handle journaling without adding it into the system) and OSX (which could handle it without any problems). Perhaps it's something the Darwin hackers should look into.

And by the way, Linux also does VFS plugins. Unless I'm mistaken, most Unices do. You can't port them straight over (though it would be so nice if you could) but it's the same concept.
You are in Soviet Russia. It is dark. Grue is likely to be eaten by YOU!
     
Milio
Grizzled Veteran
Join Date: Oct 2000
Status: Offline
Reply With Quote
Jan 18, 2001, 11:48 AM
 
Originally posted by RichardS:
Apple cannot switch to any new File System until resource forks die.

That means Classic must no longer exist before we can dump HFS+
I'm in no hurry to see Apple "dump" HFS+. When there is a filesystem that can handle as much file metadata as HFS+ and not store any metadata in the filename, then I might be willing to listen to the possiblity of a change.

One of my least favorite things about OS X is that it relies heavily on filenames to determine type and associated application.

[This message has been edited by Milio (edited 01-18-2001).]
     
Zarafa
Forum Regular
Join Date: Sep 2000
Location: Seattle, WA, USA
Status: Offline
Reply With Quote
Jan 18, 2001, 12:11 PM
 
Originally posted by RichardS:
Apple cannot switch to any new File System until resource forks die.
This is incorrect. There are other filesystems that suport arbitrary metadata and multiple "forks". NTFS is a good example; it happens to call forks "streams", but files can have mutiple streams. NTFS also supports POSIX standards such as case sensitivity *if* desired (Windows doesn't make use of this), on-the-fly compression, an encryption layer, etc.

Which is not to say that NTFS is necessarily the be-all end-all of filesystems, but it is important to note that HFS+ is not unique in really any way, and is considerably outdated by most standards.

     
SawThis
Guest
Status:
Reply With Quote
Jan 18, 2001, 12:26 PM
 
Originally posted by Milio:
One of my least favorite things about OS X is that it relies heavily on filenames to determine type and associated application.
Mac OS X uses the meta-data type and creator fields first (if they exist) and falls back to more "pedestrian" means like filename extensions, etc... only if the type/creator is not available. This make all the data that you currently have continue to work just the way you would expect, but makes import of data from other systems more palatable.

The only drawback to this approach is that if a file was created with a specific tool that is currently only available in Classic, it will launch that specific Classic tool to process that file, even if a comparable X-native (Carbon/Cocoa) tool exists. You can change this behavior on a file-by-file basis through the inspector, but I have not found a way to do it across-the-board.
     
Milio
Grizzled Veteran
Join Date: Oct 2000
Status: Offline
Reply With Quote
Jan 18, 2001, 12:41 PM
 
I know that type/creator is still available. It's one of the saving graces of OS X that they didn't abandon it altogether.

But I'm getting ready for a slew of applications which totally ignore assigning a type/creator. Apple already did this with the beta, and there are obviously quite a few unix developers who don't understand or don't like the benefits of the resource forks.
     
Emfbo
Guest
Status:
Reply With Quote
Jan 18, 2001, 12:49 PM
 
Originally posted by Milio:
I know that type/creator is still available. It's one of the saving graces of OS X that they didn't abandon it altogether.


But I'm getting ready for a slew of applications which totally ignore assigning a type/creator. Apple already did this with the beta, and there are obviously quite a few unix developers who don't understand or don't like the benefits of the resource forks.

I think you're confused. Type/creator and resource forks have almost nothing to do with each other. Anything accomplished by resource forks is handled quite nicely by bundles, which are also much easier to use with cross-platform tools and filesystems. Type/creator is a metadata element that should be stored in the same way that creation/modification date, ACLs, or other info is kept, as a "tag" by the filesystem.
     
Milio
Grizzled Veteran
Join Date: Oct 2000
Status: Offline
Reply With Quote
Jan 18, 2001, 12:58 PM
 
You're right. My brain is mushy from giving up caffeine. I'm just so used to editing the type/creator stuff with ResEdit that I let the distinction slide.
     
jblakeh1
Senior User
Join Date: Oct 2000
Location: Dallas, TX
Status: Offline
Reply With Quote
Jan 18, 2001, 01:55 PM
 
The only drawback to this approach is that if a file was created with a specific tool that is currently only available in Classic, it will launch that specific Classic tool to process that file, even if a comparable X-native (Carbon/Cocoa) tool exists. You can change this behavior on a file-by-file basis through the inspector, but I have not found a way to do it across-the-board.
It can be done in the PB, but it doesn't always 'stick,' but I'm guessing it will be fixed in the final. In the Inspector for a file, you can change the default app for the file type. I've done this where html files launch explorer, and I want them to launch a text editor.

------------------
.b
http://homepage.mac.com/jblakeh1
http://www.sit3.com
     
SYN
Senior User
Join Date: Oct 2000
Location: Paris, France
Status: Offline
Reply With Quote
Jan 18, 2001, 04:10 PM
 
XFS seems to be the best choice for where Apple is heading, since it has guaranteed I/O speeds, due to its heavy-duty use in SGi workstations. However, it doesn't handle meta-data well, le the BFS does.

Sooner or later, this has to come to OSX. I think it might come to OSX Server first, and then to Client. Time will tell.

------------------
Soyons r�alistes, demandons l'impossible.
Soyons R�alistes, Demandons l'impossible
     
PerfectlyNormalBeast
Dedicated MacNNer
Join Date: Sep 2000
Location: Medford, MA
Status: Offline
Reply With Quote
Feb 7, 2001, 04:30 AM
 
I sort of consider myself an expert on file system design, because I'm currently developing a commercial FS with embedded databases on OS X, NT/2K, and Linux.

First of all I'd like to clear something up about journaling. Journaling transacts changes to the file system structure. Basically, it writes changes twice. So it ensures that the following operations will succeed or fail without corruption:

Create File/Dir, Delete File/Dir, Grow File, Shrink File

It does not ensure the reliability of the file's contents. This means that growing a file may be a tad slower, but writing to that file is just as fast as it would be on a plain old fs. It does ensure that the FS is in a safe state at startup though.

That being said, HFS+ is a really lousy file system by any modern standard. Changing the master file table or it's appleish equivalent in any way, requires locking the whole MFT. This means that only one process, and only one thread in that process can make any of the changes listed above. On OS 9 this limitation was acceptable, but it severely hampers performance in a pre-emptive multi-threaded OS like OS X. HFS+ also uses very limited indices, something that a good FS should use plenty of. Look at BFS for an outstanding use of indices.

I'd really like to see Apple or some third party develop a journaling FS that's designed to be lightning fast. XFS is a nice FS, but has some quirks and handles metadata poorly. Maybe when I'm done with my work for my company, I'll write a killer FS for OS X. Don't hold your breath though; I'm sadly very busy here.

If you've got a question about file systems, I'll try to help.
[email protected]
     
The DJ
Grizzled Veteran
Join Date: Oct 2000
Location: Netherlands
Status: Offline
Reply With Quote
Feb 7, 2001, 05:53 AM
 
I don't understand it.
Why so much trouble about a JFS? I have been screaming of the roofs for a smb fs and nobody seems to be interested.
All i hear is "Oh, someone is bound to port that"
or "I hear some ppl are working on porting Samba"

First, yeah right. Even the *nix guru's didn't quite port it to BSD (they are working at it though, just not realy fast ), so how are we going to see a good version for OSX soon. And according to my inquiries with Developers, apple isn't working on it either at the moment. That leaves commercial. That immediatly being the problem aswell ;-)

The latter respons references to my own effort to port Samba, but that's a bit different then writing a filesystem. I don't know *(@#$* about that and no one seems to be able to help me.

Derk-Jan Hartman (Desperatly trying to get a good port of Samba to OSX.)
[email protected] http://home.student.utwente.nl/d.har...eik/samba.html
Current problems: 1: No smb filesystem
2: No -maxdepth option in find
3: No OSX native GUI interface
4: Don't know how to write a makefile that will make adaptions (settings) for OSX
5: Can't get scripts working in the OSX installer.app so you need shellscripts to install the thing.

In the workings: 2: believed to be solved in final
3: Building it myself in Cocoa/JAVA, but prob. missing some stuff, need all the help i can get. (shareware programmers of FTP clients out there?)


[This message has been edited by The DJ (edited 02-07-2001).]

Derk-Jan Hartman, Student of the University Twente (NL), developer of VLC media player
     
SJ  (op)
Junior Member
Join Date: Dec 2000
Location: Earth
Status: Offline
Reply With Quote
Feb 7, 2001, 07:30 AM
 
Originally posted by PerfectlyNormalBeast:
I sort of consider myself an expert on file system design, because I'm currently developing a commercial FS with embedded databases on OS X, NT/2K, and Linux.

I'd really like to see Apple or some third party develop a journaling FS that's designed to be lightning fast. XFS is a nice FS, but has some quirks and handles metadata poorly. Maybe when I'm done with my work for my company, I'll write a killer FS for OS X. Don't hold your breath though; I'm sadly very busy here.

If you've got a question about file systems, I'll try to help.
[email protected]
Thanks for the info PNB. It always good to hear from people who have a clue!

That said, I really hope Apple comes out with a decend FS for MacOS X because otherwise it kinda defeats the purpose of having a whiz-bang OS in the FS it is running on sucks!

I wonder who the FS guys at Apple are!
     
Zarafa
Forum Regular
Join Date: Sep 2000
Location: Seattle, WA, USA
Status: Offline
Reply With Quote
Feb 7, 2001, 01:29 PM
 
Originally posted by PerfectlyNormalBeast:

I'd really like to see Apple or some third party develop a journaling FS that's designed to be lightning fast.
Journaling per se isn't necessarily the best or only approach to the problem of inconsistent filesystems and fast recovery. Soft Updates are an alternate approach that may be worth exploring. http://www.usenix.org/publications/l...l/seltzer.html

Since FreeBSD is rolling fsck-less booting, soft updates, and other enhancements (including support for extended attributes such as ACLs) into their UFS implementation, and Darwin borrows heavily from the FreeBSD codebase, it might be best to pursue that approach for the time being.

Of course, there's nothing saying only one filesystem is appropriate. The right tool for the right job, and all that.

     
Millennium
Clinically Insane
Join Date: Nov 1999
Status: Offline
Reply With Quote
Feb 7, 2001, 01:48 PM
 
Personally, I'd love to see a BeFS port.

It's got journaling, which is very important. But it also has one extremely important feature most other systems lack: metadata support.

Using this, it would be very simple to create work-alikes for the best parts of HFS+: type/creators, resource forks, and true aliases. BeOS, in fact, can already handle this stuff. The end result would be something that would feel almost exactly like HFS+ to the user, except that it'd be somewhat faster and you couldn't boot Classic off of it.

I've been working on getting documentation for the filesystem; it certainly seems like a worthwhile port for OSX. All of HFS+'s strengths, none of its weaknesses, and still a very Mac-like filesystem. Anyone else have opinions?
You are in Soviet Russia. It is dark. Grue is likely to be eaten by YOU!
     
PerfectlyNormalBeast
Dedicated MacNNer
Join Date: Sep 2000
Location: Medford, MA
Status: Offline
Reply With Quote
Feb 7, 2001, 01:57 PM
 
Originally posted by Zarafa:
Journaling per se isn't necessarily the best or only approach to the problem of inconsistent filesystems and fast recovery. Soft Updates are an alternate approach that may be worth exploring. [URL=http://www.usenix.org/publications/library/proceedings/usenix2000/general/seltzer.html]http://www.usenix.org/publications/library/proceedings/usenix2000/general/seltzer.html[/ URL]
I've read about a couple of other options for maintaining consistency. I haven't seen this paper before, but I've done a bit of research into the ordered write technique. I could be mistaken, but I've found that this method tends to limit concurrency in the system. Granted, even journaling serializes disk writes for metadata info at some level. When I say metadata here, I'm refering to disk structure, not resource forks. I've just found that a journaling system let's you put the blocks you are writing in any order. By writing blocks in the order they appear on disk, you can often double the speed of your I/O. And since writing to the log is often one contiguous write, that happens very fast as well.
     
WickedDyno
Guest
Status:
Reply With Quote
Feb 7, 2001, 02:01 PM
 
The slashdot denizens might be opinionated and pigheaded, but at least they know something.
A little knowledge is a dangerous thing. Slashdot demonstrates this amply.

The DJ: Samba is already ported to the Public Beta.

Two things that need to be done: Classic should be able to boot off a disk image. And someone should port ext2 and ReiserFS support to OS X so you can access Linux disks on your machine.

------------------
Gravity is the enemy.
     
mitchell_pgh
Posting Junkie
Join Date: Feb 2000
Location: Washington, DC
Status: Offline
Reply With Quote
Feb 7, 2001, 02:06 PM
 
IBM has just open sourced it's TTFS, an advanced journaling file system that blows the doors off of the competition (NTFS, UFS, HFS+, BeFS, etc). It contains all of the strengths of BeFS [type:creators, true aliases and resource forks] and various other FS's, but it also provides ease 256 bit FS level strong encryption (this would slow the machine slightly [app. 5% from their reports])

go to http://www.ibm.com/news for reports
     
PerfectlyNormalBeast
Dedicated MacNNer
Join Date: Sep 2000
Location: Medford, MA
Status: Offline
Reply With Quote
Feb 7, 2001, 02:06 PM
 
Originally posted by Millennium:
Personally, I'd love to see a BeFS port.

I've been working on getting documentation for the filesystem; it certainly seems like a worthwhile port for OSX. All of HFS+'s strengths, none of its weaknesses, and still a very Mac-like filesystem. Anyone else have opinions?
Try Practical File System Design with the Be File System by Dominic Giampaolo. ISBN: 1558604979

It's not a full spec. It's more of a guide to writing an FS with the BFS in mind. It does go into a bit of details about the structures they use and lists some pseudocode for working with those structures. With some effort you could clone the system. However, I do believe that the BFS belongs to Be, and I'm not sure how they feel about ports. You'd have to ask them. Luckily, they're one of the few OS companies that will answer the phone with a person.
     
The DJ
Grizzled Veteran
Join Date: Oct 2000
Location: Netherlands
Status: Offline
Reply With Quote
Feb 7, 2001, 03:46 PM
 
Originally posted by WickedDyno:
A little knowledge is a dangerous thing. Slashdot demonstrates this amply.
The DJ: Samba is already ported to the Public Beta.
typical response to my questions.
It seems you are guilty of your own first remark.
First the smb filesystem is something completly different then the Samba source. Secondly as i said in the post, that port is MINE, the article at MacNN, where it premiered, is my contribution. I did the work on it. It's the only port as far as i know and although it works in most cases, still much work needs to be done.

It isn't officially is in the Samba sourcetree.
There is a bug in smbclient with put'ing directories and non of the more advanced stuff like smbmount, smbwrappers etc work.
And although not part of Samba, there is no smbfs, which is essential if you want to mount Samba shares.

DJ

[This message has been edited by The DJ (edited 02-07-2001).]

Derk-Jan Hartman, Student of the University Twente (NL), developer of VLC media player
     
   
Thread Tools
 
Forum Links
Forum Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts
BB code is On
Smilies are On
[IMG] code is On
HTML code is Off
Top
Privacy Policy
All times are GMT -4. The time now is 05:38 AM.
All contents of these forums © 1995-2017 MacNN. All rights reserved.
Branding + Design: www.gesamtbild.com
vBulletin v.3.8.8 © 2000-2017, Jelsoft Enterprises Ltd.,