Forgot your password?

dutky's Journal: checkpointed files

Journal by dutky

Just some thoughts on an abstract checkpointed file library:

The interest in checkpointing comes from the desire to preserve a known good state of an actively read or written file in the face of unpredictable system failure. We would like to be able to resume processing from the last known good point.

It seems that we only really need a small set of operations to support checkpointed files:

First, we need routines to open and close a checkpointed file:

CP_FILE *cp_open(char *fname, char *mode);

int cp_close(CP_FILE *cpf);

Next, we need routines to read from and write to the checkpointed file (since we don't have a regular file pointer in hand, we can't use fread or fwrite):

int cp_read(CP_FILE *cpf, void *buf, int size, int count);

int cp_write(CP_FILE *cpf, void *buf, int size, int count);

Finally we need a routine to update the file's position in the checkpoint store and flush the file buffers to disk:

int cp_check(CP_FILE *cpf);

The cp_open routine will either open an existing checkpointed file (if one with the given name is found in the checkpoint store) or create a new file and make an entry for the file in the checkpoint store.

The cp_close routine will mark the file's entry in the checkpoint store as completed and close the open file.

We may want a few routines to round out the support for file operations:

fpos_t cp_tell(CP_FILE *cpf);

int cp_seek(CP_FILE *cpf, fpos_t fpos);

We also need some routines to manage the checkpointing system, specifically to adapt the framework to different implementations of the checkpoint store:

int cp_hooks(int (*ins)(char *fname, void **userdata), int (*upd)(char *fname, fpos_t fpos, void *userdata), int (*qry)(char *fname, fpos_t *fpos, void *userdata), int (*com)(void), int (*rbk)(void), int (*ini)(void), int (*fin)(void));

The ins hook is called when a new entry is needed in the checkpoint store for a named file. The insert routine may return a pointer to a block of user data.

The upd hook is called to update the file position value in an entry. The user data block is also provided.

The qry hook is used to query the checkpoint store for an entry for the named file. If such an entry exists the file position and user data are returned.

The com and rbk hooks are used to commit and roll-back changes to the checkpoint store.

The ini and fin hooks are used to initialize and finnish any connection to the checkpoint store.

If what they've been doing hasn't solved the problem, tell them to do something else. -- Gerald Weinberg, "The Secrets of Consulting"