Mailing List Archive

Hi,

On Wed, 20 Oct 1999 09:54:42 +0100, CEINTEC i+d <i+d@ceintec.drago.net>
said:

> Thanks, I'm reading now, there is some interesting things. The
> Phillipp's nmbd sounds well, but somebody can explain me, what does he
> mean by "journaling file system"?, I don't understand this term It's
> the same that a transactional file system?

Yes --- or rather, journaling is one method by which we can make a
filesystem transactional.

> or It's a file system with permanent write operations?

A filesystem without permanent writes is of rather limited use. :)

--Stephen

On Wed, Oct 20, 1999 at 05:46:10PM +0100, Stephen C. Tweedie wrote:
>Yes --- or rather, journaling is one method by which we can make a
>filesystem transactional.

As I understand it, the journaling in ext3 (which is a fantastic
design, BTW), only handles journaling of file-system meta-data,
not of actual file data. This is a common misconception I've run
into is that it handles the file data as well.

The problem is that in the event of a crash, there's little a file-system
can do to prevent the crashed applications data from being left in an
unknown state.

Sean
--
I used to think that the brain was the most wonderful organ in
my body. Then I realized who was telling me this. -- Emo Phillips
Sean Reifschneider, Inimitably Superfluous <jafo@tummy.com>
URL: <http://www.tummy.com/xvscan> HP-UX/Linux/FreeBSD/BSDOS scanning software.

Sean Reifschneider wrote:

> On Wed, Oct 20, 1999 at 05:46:10PM +0100, Stephen C. Tweedie wrote:
> >Yes --- or rather, journaling is one method by which we can make a
> >filesystem transactional.
>
> As I understand it, the journaling in ext3 (which is a fantastic
> design, BTW), only handles journaling of file-system meta-data,
> not of actual file data. This is a common misconception I've run
> into is that it handles the file data as well.

Bzzt. As a matter of fact, all it does right now is journal both data and
metadata. Stephen mentioned that we could try turning off data logging (thereby
improving vastly the performance) but that it certainly hadn't been tested.

>
> The problem is that in the event of a crash, there's little a file-system
> can do to prevent the crashed applications data from being left in an
> unknown state.
>

Yup. Even with data logging, who is to say that the last write you did to a file
actually leaves your file in a usable state, for example? I believe that's why
Alan is so high on RAID-1+NBD - the data is written to all nodes simultaneously,
so unless your entire cluster crashes, at least someone has the right data.
Again, not super-performing (and sct has mentioned that there are tweaks we can
use) but pretty doggone stable.

- Pete

Hi,

On Wed, 20 Oct 1999 11:00:24 -0600, Sean Reifschneider
<jafo@tummy.com> said:

> On Wed, Oct 20, 1999 at 05:46:10PM +0100, Stephen C. Tweedie wrote:
>> Yes --- or rather, journaling is one method by which we can make a
>> filesystem transactional.

> As I understand it, the journaling in ext3 (which is a fantastic
> design, BTW), only handles journaling of file-system meta-data,
> not of actual file data.

Actually, it journals both for now. That simplifies the journaling
core quite a bit, and I'm keeping the metadata-only journaling
disabled until I'm sure that the core journal code is rock-solid.

Full journaling will still be an option in the long term, though. In
particular, for systems such as NFS servers, fast commit of written
data to disk is necessary for low latency, and by allowing data to be
journaled to a separate journal device, we can improve NFS write
latencies enormously while retaining synchronous writes.

> This is a common misconception I've run into is that it handles the
> file data as well.

It does.

> The problem is that in the event of a crash, there's little a file-system
> can do to prevent the crashed applications data from being left in an
> unknown state.

Absolutely. Data journaling is not application journaling. You can
ensure that the data is intact on disk, but that doesn't mean that it
represents a consistent state for the application. Only the
application can ever have a hope of understanding what its own
transaction semantics are.

What filesystem data journaling _can_ offer in this situation is fast
commit of filesystem transactions to disk. This is especially true if
you have a separate journal disk, but even without that, committing
data to disk (ie. O_SYNC writes or fsync()) becomes a simple matter of
writing a single sequential record to the journal. The filesystem can
propagate the in-place updates to the main on-disk structures later on
at leisure, but the initial synchronous data write can be done much
more quickly with filesystem data journaling.

--Stephen

"Stephen C. Tweedie" wrote:

> Absolutely. Data journaling is not application journaling. You can
> ensure that the data is intact on disk, but that doesn't mean that it
> represents a consistent state for the application. Only the
> application can ever have a hope of understanding what its own
> transaction semantics are.

I am very interested in process checkpoint and rollback. I wrote my
dissertation and several papers on optimistic programming several years ago (
http://www.cse.ogi.edu/~crispin/hope.html ). Optimistic programming depends
on being able to checkpoint a process now, and roll back to that state later.

I implemented a crude form of checkpoint and rollback for processes, but
couldn't do anything about rolling back file state. Can a journaling file
system give me the hooks I need to be able to checkpoint and rollback the
state of one or more files?

Thanks,
Crispin
-----
Crispin Cowan, CTO, WireX Communications, Inc. http://wirex.com
Free Hardened Linux Distribution: http://immunix.org

Hi,

On Thu, 21 Oct 1999 20:05:33 +0000, Crispin Cowan <crispin@cse.ogi.edu>
said:

> I am very interested in process checkpoint and rollback. ...

> I implemented a crude form of checkpoint and rollback for processes,
> but couldn't do anything about rolling back file state. Can a
> journaling file system give me the hooks I need to be able to
> checkpoint and rollback the state of one or more files?

No. The journal does not keep any form of snapshot other than a record
of the immediately outstanding IO pending on the filesystem. It would
break the journal semantics completely to have any form of long-term
snapshot in the log.

Snapshots are still useful to have, but implementing them is an
orthogonal issue to journaling.

--Stephen