Day: February 25, 2015

On touch’s use of futimes() and a filesystem bug.

February 25, 2015 C/C++ development and debugging. , , , , ,

In my last post, Why does touch include a utimensat() syscall?, I’d pointed out that strace reports an utimensat syscall from touch, and noted that this appeared to have the effect of clearing the subsecond portion of the files mtime.

It turns out that strace was slightly lying, and we actually have a call to futimens( fd, NULL ).  I was able to see that by debugging into coreutils’s touch.c and see what it is doing.  The purpose of this futimens() syscall is supposed to be to set the time to the current time.  It appears that clearcase V8 + MVFS + futimes() doesn’t respect the microsecond granular times that were implemented in V8.   This API, as currently implemented will set the file’s mtime to the current time, but only respects the seconds portion of that time.

With clearcase V8 MVFS view private files having the capability for subsecond granularity, the end result is that this pushes the modification time backwards if you get unlucky enough to execute the futimes() in the same second as the file creation.  That is almost always.  It can be seen to work correctly if you put a long enough sleep in the code between the initial creation point for one file, and the “touch” of the second.  Here’s a bit of standalone code that illustrates:

#include <sys/types.h>
#include <sys/stat.h>
#include <fcntl.h>
#include <getopt.h>
#include <errno.h>
#include <stdio.h>
#include <string.h>
#include <unistd.h>
#include <stdlib.h>
#include <time.h>

int main( int argc, char ** argv )
{
   int c                = 0 ;
   long sleepTimeUsec    = 10000 ;
   int fd1                = -1 ;
   int fd2                = -1 ;
   char * err             = NULL ;
   int rc                = 0 ;
   struct timespec ts = { 0, 0 } ;

   while ( (c = getopt( argc, argv, "s:" )) != EOF )
   {
      if ( c == 's' )
      {
         sleepTimeUsec = strtol( optarg, &err, 10 ) ;
         if ( err[0] )
         {
            fprintf( stderr, "unexpected input in -s parameter '%s'\n",
                     optarg ) ;
            return 1 ;
         }
      }
   }

   unlink( "touchFirst" ) ;
   unlink( "touchSecond" ) ;

   fd1 = open( "touchFirst", O_CREAT ) ;
   if ( -1 == fd1 )
   {
      fprintf( stderr, "create: 'touchFirst' failed: %d (%s)\n", 
               errno, strerror( errno ) ) ;
      return 2 ;
   }
   close( fd1 ) ;

   fd2 = open( "touchSecond", O_CREAT ) ;
   if ( -1 == fd2 )
   {
      fprintf( stderr, "create: 'touchSecond' failed: %d (%s)\n", 
               errno, strerror( errno ) ) ;
      return 3 ;
   }

   ts.tv_sec = sleepTimeUsec / 1000000 ;
   ts.tv_nsec = (sleepTimeUsec % 1000000)*1000 ;
   rc = nanosleep( &ts, NULL ) ;
   if ( -1 == rc )
   {
      fprintf( stderr, "nanosleep(%lu,%lu) failed: %d (%s)\n", 
               (unsigned long)ts.tv_sec, (unsigned long)ts.tv_nsec,
               errno, strerror( errno ) ) ;
      return 4 ;
   }

   rc = futimens( fd2, NULL ) ;
   if ( -1 == rc )
   {
      fprintf( stderr, "futimens failed: %d (%s)\n", 
               errno, strerror( errno ) ) ;
      return 5 ;
   }

   close( fd2 ) ;

   return 0 ;
}

Here’s a pair of calls that illustrate the bug.

/vbs/engn/t2> a.out -s 500000 ;  ls --full-time touch*
-r-x--xr-- 1 peeterj pdxdb2 0 2015-02-25 23:16:36.498049000 -0500 touchFirst
-r-x--xr-- 1 peeterj pdxdb2 0 2015-02-25 23:16:36.000000000 -0500 touchSecond
/vbs/engn/t2> a.out -s 500000 ;  ls --full-time touch*
-r-x--xr-- 1 peeterj pdxdb2 0 2015-02-25 23:16:38.900498000 -0500 touchFirst
-r-x--xr-- 1 peeterj pdxdb2 0 2015-02-25 23:16:39.000000000 -0500 touchSecond

The first is the buggy call, and the time goes backwards despite a half second sleep. The second call is okay, because 0.9+0.5 of a second ends up in the next second, so the second “touched” file has a timestamp after the creation of the second file (as expected.)

Why does touch include a utimensat() syscall?

February 25, 2015 perl and general scripting hackery , , , , , , , , ,

I’m seeing odd time sequencing of files when using clearcase version 8 dynamic views, which makes me wonder about an aspect of the (gnu) touch command. Running:

> cat u
rm -f touchedEarlier touchedLater

perl -e "open (F, '> touchedEarlier') || die"
touch touchedLater

ls --full-time touchedEarlier touchedLater

produces:

> ./u
-rw-r--r-- 1 peeterj pdxdb2 0 2015-02-25 11:42:05.833044000 -0500 touchedEarlier
-rw-r--r-- 1 peeterj pdxdb2 0 2015-02-25 11:42:05.000000000 -0500 touchedLater

Notice that the file that is touched by doing a perl “open” ends up with a later time, despite the fact that it was done logically earlier than the touch.

Running this command outside of a clearcase dynamic view shows zeros only in the subsecond times (also the behaviour of clearcase V7). Needless to say, this difference in file times from their creation sequence wreaks havoc on make.

I was curious how the two touch methods differed, and stracing them shows that the touch differs by including a utimesat() syscall. The perl touch is:

open(“touchedEarlier”, O_WRONLY|O_CREAT|O_TRUNC, 0666) = 3
ioctl(3, SNDCTL_TMR_TIMEBASE or TCGETS, 0x7fff29cd29f0) = -1 ENOTTY (Inappropriate ioctl for device)
lseek(3, 0, SEEK_CUR) = 0
fstat(3, {st_mode=S_IFREG|0644, st_size=0, …}) = 0
fcntl(3, F_SETFD, FD_CLOEXEC) = 0
close(3) = 0
exit_group(0) = ?

whereas the touch command has:

open(“touchedLater”, O_WRONLY|O_CREAT|O_NOCTTY|O_NONBLOCK, 0666) = 3
dup2(3, 0) = 0
close(3) = 0
dup2(0, 0) = 0
utimensat(0, NULL, NULL, 0) = 0
close(0) = 0
close(1) = 0
close(2) = 0
exit_group(0) = ?

It appears that the touch command explicitly zeros the subsecond portion of the files’ timestamp.

I also see that perl’s File::Touch module does the same thing, but uses a different mechanism. I see the following in a strace of such a Touch() call:

stat(“xxyyzz”, 0x656060)                = -1 ENOENT (No such file or directory)
open(“xxyyzz”, O_WRONLY|O_CREAT|O_NOCTTY|O_NONBLOCK, 0666) = 3
ioctl(3, SNDCTL_TMR_TIMEBASE or TCGETS, 0x7fff16bb3c60) = -1 ENOTTY (Inappropriate ioctl for device)
lseek(3, 0, SEEK_CUR)                   = 0
fstat(3, {st_mode=S_IFREG|0644, st_size=0, …}) = 0
fcntl(3, F_SETFD, FD_CLOEXEC)           = 0
close(3)                                = 0
utimes(“xxyyzz”, {{1424884553, 0}, {1424884553, 0}}) = 0

I am very curious why touch and perl’s File::Touch() both explicitly zero the subsecond modification time for the file (using utimensat() or utimes() syscalls)?