C/C++ development and debugging.

Accidental correctness

October 6, 2015 C/C++ development and debugging. ,

Here’s a fun warning message I saw today:


foo.h:408:15: warning: & has lower precedence than !=; != will be evaluated first [-Wparentheses]
  if( nibbles & 0x01 != 0 )
              ^~~~~~~~~~~
foo.h:408:15: note: place parentheses around the '!=' expression to silence this warning
  if( nibbles & 0x01 != 0 )
              ^
                (        )
foo.h:408:15: note: place parentheses around the & expression to evaluate it first
  if( nibbles & 0x01 != 0 )
              ^
      (             )

The author of the code appears to fluked out with this statement, since (0x01 != 0) == 1, which is exactly the bit that they were attempting to test against. Ironically, the != 0 was probably added for clarity, but if they had wanted to test against any other bit, the code wouldn’t have done what they expected, and “clarifying the statement” would have bugified it.

extern vs const in C++ and C code.

October 5, 2015 C/C++ development and debugging. , , , , , ,

We now build DB2 on linux ppcle with the IBM xlC 13.1.2 compiler. This version of the compiler is a hybrid compared to any previous compilers, retaining the IBM xlC backend for power, but using the clang front end. Because of this we are exposed to a large number of warnings that we don’t see with many other compilers (well we probably do for our MacOSX port, but we do not really have active development on that platform at the moment), and I’ve been trying to take down those counts to manageable levels. Header files that produce warnings have been my first target since they introduce the most repeated noise.

One message that I was seeing hundreds of was

warning: 'extern' variable has an initializer [-Wextern-initializer]

This seemed to be coming from headers that did something like:

#if defined FOO_INITIALIZE_IT_IN_SOME_SOURCE_FILE
extern const TYPE foo[] = { ... } ;
#else
extern const TYPE foo[] ;
#endif

where FOO_INITIALIZE_IT_IN_SOME_SOURCE_FILE is defined at the top of a source file that explicitly includes this header. My attempt to handle the messages was to remove the ‘extern’ from the initialization case, but I was suprised to see link errors as a result of some of those changes. It turns out that there are some subtle differences between different variations of const and extern with an array declaration of this sort. Here’s a bit of sample code:

// t.h
extern const int x[] ;
extern int y[] ;
extern int z[] ;


// t.cc
#if defined WANT_LINK_ERROR
const int x[] = { 42 } ;
#else
extern const int x[] = { 42 } ;
#endif

extern int y[] = { 42 } ;
int z[] = { 42 } ;

When WANT_LINK_ERROR isn’t defined, this produces just one clang warning message

t.cc:8:12: warning: 'extern' variable has an initializer [-Wextern-initializer]
extern int y[] = { 42 } ;
           ^

Note that the ‘extern const’ has no such warning, nor does the non-const symbol that’s been declared ‘extern’ in the header. However, removing the extern from the const case (via -DWANT_LINK_ERROR) results in no symbol ‘x’ available to other consumers. The extern is required for const symbols, but generates a warning for non-const symbols.

It appears that this is also C++ specific. A const symbol in C compiled code is available for external use, regardless of whether extern is used:



$ clang -c t.c
t.c:5:18: warning: 'extern' variable has an initializer [-Wextern-initializer]
extern const int x[] = { 42 } ;
                 ^
t.c:8:12: warning: 'extern' variable has an initializer [-Wextern-initializer]
extern int y[] = { 42 } ;
           ^
2 warnings generated.

$ nm t.o
0000000000000000 R x
0000000000000000 D y
0000000000000004 D z

$ clang -c -DWANT_LINK_ERROR t.c
t.c:8:12: warning: 'extern' variable has an initializer [-Wextern-initializer]
extern int y[] = { 42 } ;
           ^
1 warning generated.
$  nm t.o
0000000000000000 R x
0000000000000000 D y
0000000000000004 D z

whereas that same symbol requires extern if it is const in C++:


$ clang++ -c t.cc
t.cc:8:12: warning: 'extern' variable has an initializer [-Wextern-initializer]
extern int y[] = { 42 } ;
           ^
1 warning generated.
$ nm t.o
0000000000000000 R x
0000000000000000 D y
0000000000000004 D z



$ clang++ -c -DWANT_LINK_ERROR t.cc
t.cc:8:12: warning: 'extern' variable has an initializer [-Wextern-initializer]
extern int y[] = { 42 } ;
           ^
1 warning generated.
$ nm t.o
0000000000000000 D y
0000000000000004 D z

I hadn’t expected the const to interact this way with extern. I am guessing that C++ allows for the compiler to not generate symbols for global scope const variables, unless you ask for that by using extern, whereas with C you get the symbol like-it-or-not. This particular message from the clang front end is only for non-const extern initializations, making across the board fixing of messages for extern initialization of the sort above trickier. This makes it so that you can’t do an across the board replacement of extern in initializers for a given file without first ensuring that the symbol isn’t const. It looks like dealing with this will have to be done much more carefully than I first tried.

Debugging with optimization: -g8

May 8, 2015 C/C++ development and debugging. , , , , ,

It was suggested to me to try -g8 for debugging some optimized code (xlC compiler, linuxppcle platform).  Turns out that there is a whole set of -gN options with the xlC compiler that allow various levels of debug instrumentation.

For the optimization bug that I was debugging, -g8 made the problem go away, but with -g5 I was able to break at the nearest case: block and determine that the data in the variables was good coming into that case statement.  Since what appears to come out of that case statement is bad, this isolates the problem significantly.

Hacking header file timestamps

May 7, 2015 C/C++ development and debugging. , , ,

Suppose you change a header foo.h, but your build system has dependency checking that will force the world to be rebuilt. A handy way to avoid that is with judicious use of the touch command, such as:

touch -t197301010000 foo.h

or (on Linux)

touch --reference another-file-that-is-early-enough-in-time foo.h

Both of these can be used to force the modification time for the file backwards.

This can be very useful way to save time.

It is also a great way to not build enough or break the build if misused!

I’d recommend this only for prototyping type work, to be followed up with a subsequent touch to restore the timestamps to normal, and then do a proper build.

On touch’s use of futimes() and a filesystem bug.

February 25, 2015 C/C++ development and debugging. , , , , ,

In my last post, Why does touch include a utimensat() syscall?, I’d pointed out that strace reports an utimensat syscall from touch, and noted that this appeared to have the effect of clearing the subsecond portion of the files mtime.

It turns out that strace was slightly lying, and we actually have a call to futimens( fd, NULL ).  I was able to see that by debugging into coreutils’s touch.c and see what it is doing.  The purpose of this futimens() syscall is supposed to be to set the time to the current time.  It appears that clearcase V8 + MVFS + futimes() doesn’t respect the microsecond granular times that were implemented in V8.   This API, as currently implemented will set the file’s mtime to the current time, but only respects the seconds portion of that time.

With clearcase V8 MVFS view private files having the capability for subsecond granularity, the end result is that this pushes the modification time backwards if you get unlucky enough to execute the futimes() in the same second as the file creation.  That is almost always.  It can be seen to work correctly if you put a long enough sleep in the code between the initial creation point for one file, and the “touch” of the second.  Here’s a bit of standalone code that illustrates:

#include <sys/types.h>
#include <sys/stat.h>
#include <fcntl.h>
#include <getopt.h>
#include <errno.h>
#include <stdio.h>
#include <string.h>
#include <unistd.h>
#include <stdlib.h>
#include <time.h>

int main( int argc, char ** argv )
{
   int c                = 0 ;
   long sleepTimeUsec    = 10000 ;
   int fd1                = -1 ;
   int fd2                = -1 ;
   char * err             = NULL ;
   int rc                = 0 ;
   struct timespec ts = { 0, 0 } ;

   while ( (c = getopt( argc, argv, "s:" )) != EOF )
   {
      if ( c == 's' )
      {
         sleepTimeUsec = strtol( optarg, &err, 10 ) ;
         if ( err[0] )
         {
            fprintf( stderr, "unexpected input in -s parameter '%s'\n",
                     optarg ) ;
            return 1 ;
         }
      }
   }

   unlink( "touchFirst" ) ;
   unlink( "touchSecond" ) ;

   fd1 = open( "touchFirst", O_CREAT ) ;
   if ( -1 == fd1 )
   {
      fprintf( stderr, "create: 'touchFirst' failed: %d (%s)\n", 
               errno, strerror( errno ) ) ;
      return 2 ;
   }
   close( fd1 ) ;

   fd2 = open( "touchSecond", O_CREAT ) ;
   if ( -1 == fd2 )
   {
      fprintf( stderr, "create: 'touchSecond' failed: %d (%s)\n", 
               errno, strerror( errno ) ) ;
      return 3 ;
   }

   ts.tv_sec = sleepTimeUsec / 1000000 ;
   ts.tv_nsec = (sleepTimeUsec % 1000000)*1000 ;
   rc = nanosleep( &ts, NULL ) ;
   if ( -1 == rc )
   {
      fprintf( stderr, "nanosleep(%lu,%lu) failed: %d (%s)\n", 
               (unsigned long)ts.tv_sec, (unsigned long)ts.tv_nsec,
               errno, strerror( errno ) ) ;
      return 4 ;
   }

   rc = futimens( fd2, NULL ) ;
   if ( -1 == rc )
   {
      fprintf( stderr, "futimens failed: %d (%s)\n", 
               errno, strerror( errno ) ) ;
      return 5 ;
   }

   close( fd2 ) ;

   return 0 ;
}

Here’s a pair of calls that illustrate the bug.

/vbs/engn/t2> a.out -s 500000 ;  ls --full-time touch*
-r-x--xr-- 1 peeterj pdxdb2 0 2015-02-25 23:16:36.498049000 -0500 touchFirst
-r-x--xr-- 1 peeterj pdxdb2 0 2015-02-25 23:16:36.000000000 -0500 touchSecond
/vbs/engn/t2> a.out -s 500000 ;  ls --full-time touch*
-r-x--xr-- 1 peeterj pdxdb2 0 2015-02-25 23:16:38.900498000 -0500 touchFirst
-r-x--xr-- 1 peeterj pdxdb2 0 2015-02-25 23:16:39.000000000 -0500 touchSecond

The first is the buggy call, and the time goes backwards despite a half second sleep. The second call is okay, because 0.9+0.5 of a second ends up in the next second, so the second “touched” file has a timestamp after the creation of the second file (as expected.)