C/C++ development and debugging.

extern vs const in C++ and C code.

October 5, 2015 C/C++ development and debugging. , , , , , ,

We now build DB2 on linux ppcle with the IBM xlC 13.1.2 compiler. This version of the compiler is a hybrid compared to any previous compilers, retaining the IBM xlC backend for power, but using the clang front end. Because of this we are exposed to a large number of warnings that we don’t see with many other compilers (well we probably do for our MacOSX port, but we do not really have active development on that platform at the moment), and I’ve been trying to take down those counts to manageable levels. Header files that produce warnings have been my first target since they introduce the most repeated noise.

One message that I was seeing hundreds of was

warning: 'extern' variable has an initializer [-Wextern-initializer]

This seemed to be coming from headers that did something like:

#if defined FOO_INITIALIZE_IT_IN_SOME_SOURCE_FILE
extern const TYPE foo[] = { ... } ;
#else
extern const TYPE foo[] ;
#endif


where FOO_INITIALIZE_IT_IN_SOME_SOURCE_FILE is defined at the top of a source file that explicitly includes this header. My attempt to handle the messages was to remove the ‘extern’ from the initialization case, but I was suprised to see link errors as a result of some of those changes. It turns out that there are some subtle differences between different variations of const and extern with an array declaration of this sort. Here’s a bit of sample code:

// t.h
extern const int x[] ;
extern int y[] ;
extern int z[] ;


// t.cc
#if defined WANT_LINK_ERROR
const int x[] = { 42 } ;
#else
extern const int x[] = { 42 } ;
#endif

extern int y[] = { 42 } ;
int z[] = { 42 } ;


When WANT_LINK_ERROR isn’t defined, this produces just one clang warning message

t.cc:8:12: warning: 'extern' variable has an initializer [-Wextern-initializer]
extern int y[] = { 42 } ;
           ^

Note that the ‘extern const’ has no such warning, nor does the non-const symbol that’s been declared ‘extern’ in the header. However, removing the extern from the const case (via -DWANT_LINK_ERROR) results in no symbol ‘x’ available to other consumers. The extern is required for const symbols, but generates a warning for non-const symbols.

It appears that this is also C++ specific. A const symbol in C compiled code is available for external use, regardless of whether extern is used:



$ clang -c t.c
t.c:5:18: warning: 'extern' variable has an initializer [-Wextern-initializer]
extern const int x[] = { 42 } ;
                 ^
t.c:8:12: warning: 'extern' variable has an initializer [-Wextern-initializer]
extern int y[] = { 42 } ;
           ^
2 warnings generated.

$ nm t.o
0000000000000000 R x
0000000000000000 D y
0000000000000004 D z

$ clang -c -DWANT_LINK_ERROR t.c
t.c:8:12: warning: 'extern' variable has an initializer [-Wextern-initializer]
extern int y[] = { 42 } ;
           ^
1 warning generated.
$  nm t.o
0000000000000000 R x
0000000000000000 D y
0000000000000004 D z


whereas that same symbol requires extern if it is const in C++:


$ clang++ -c t.cc
t.cc:8:12: warning: 'extern' variable has an initializer [-Wextern-initializer]
extern int y[] = { 42 } ;
           ^
1 warning generated.
$ nm t.o
0000000000000000 R x
0000000000000000 D y
0000000000000004 D z



$ clang++ -c -DWANT_LINK_ERROR t.cc
t.cc:8:12: warning: 'extern' variable has an initializer [-Wextern-initializer]
extern int y[] = { 42 } ;
           ^
1 warning generated.
$ nm t.o
0000000000000000 D y
0000000000000004 D z


I hadn’t expected the const to interact this way with extern. I am guessing that C++ allows for the compiler to not generate symbols for global scope const variables, unless you ask for that by using extern, whereas with C you get the symbol like-it-or-not. This particular message from the clang front end is only for non-const extern initializations, making across the board fixing of messages for extern initialization of the sort above trickier. This makes it so that you can’t do an across the board replacement of extern in initializers for a given file without first ensuring that the symbol isn’t const. It looks like dealing with this will have to be done much more carefully than I first tried.

Debugging with optimization: -g8

May 8, 2015 C/C++ development and debugging. , , , , ,

It was suggested to me to try -g8 for debugging some optimized code (xlC compiler, linuxppcle platform).  Turns out that there is a whole set of -gN options with the xlC compiler that allow various levels of debug instrumentation.

For the optimization bug that I was debugging, -g8 made the problem go away, but with -g5 I was able to break at the nearest case: block and determine that the data in the variables was good coming into that case statement.  Since what appears to come out of that case statement is bad, this isolates the problem significantly.

Hacking header file timestamps

May 7, 2015 C/C++ development and debugging. , , ,

Suppose you change a header foo.h, but your build system has dependency checking that will force the world to be rebuilt. A handy way to avoid that is with judicious use of the touch command, such as:

touch -t197301010000 foo.h 

or (on Linux)

touch --reference another-file-that-is-early-enough-in-time foo.h

Both of these can be used to force the modification time for the file backwards.

This can be very useful way to save time.

It is also a great way to not build enough or break the build if misused!

I’d recommend this only for prototyping type work, to be followed up with a subsequent touch to restore the timestamps to normal, and then do a proper build.

On touch’s use of futimes() and a filesystem bug.

February 25, 2015 C/C++ development and debugging. , , , , ,

In my last post, Why does touch include a utimensat() syscall?, I’d pointed out that strace reports an utimensat syscall from touch, and noted that this appeared to have the effect of clearing the subsecond portion of the files mtime.

It turns out that strace was slightly lying, and we actually have a call to futimens( fd, NULL ).  I was able to see that by debugging into coreutils’s touch.c and see what it is doing.  The purpose of this futimens() syscall is supposed to be to set the time to the current time.  It appears that clearcase V8 + MVFS + futimes() doesn’t respect the microsecond granular times that were implemented in V8.   This API, as currently implemented will set the file’s mtime to the current time, but only respects the seconds portion of that time.

With clearcase V8 MVFS view private files having the capability for subsecond granularity, the end result is that this pushes the modification time backwards if you get unlucky enough to execute the futimes() in the same second as the file creation.  That is almost always.  It can be seen to work correctly if you put a long enough sleep in the code between the initial creation point for one file, and the “touch” of the second.  Here’s a bit of standalone code that illustrates:

#include <sys/types.h>
#include <sys/stat.h>
#include <fcntl.h>
#include <getopt.h>
#include <errno.h>
#include <stdio.h>
#include <string.h>
#include <unistd.h>
#include <stdlib.h>
#include <time.h>

int main( int argc, char ** argv )
{
   int c                = 0 ;
   long sleepTimeUsec    = 10000 ;
   int fd1                = -1 ;
   int fd2                = -1 ;
   char * err             = NULL ;
   int rc                = 0 ;
   struct timespec ts = { 0, 0 } ;

   while ( (c = getopt( argc, argv, "s:" )) != EOF )
   {
      if ( c == 's' )
      {
         sleepTimeUsec = strtol( optarg, &err, 10 ) ;
         if ( err[0] )
         {
            fprintf( stderr, "unexpected input in -s parameter '%s'\n",
                     optarg ) ;
            return 1 ;
         }
      }
   }

   unlink( "touchFirst" ) ;
   unlink( "touchSecond" ) ;

   fd1 = open( "touchFirst", O_CREAT ) ;
   if ( -1 == fd1 )
   {
      fprintf( stderr, "create: 'touchFirst' failed: %d (%s)\n", 
               errno, strerror( errno ) ) ;
      return 2 ;
   }
   close( fd1 ) ;

   fd2 = open( "touchSecond", O_CREAT ) ;
   if ( -1 == fd2 )
   {
      fprintf( stderr, "create: 'touchSecond' failed: %d (%s)\n", 
               errno, strerror( errno ) ) ;
      return 3 ;
   }

   ts.tv_sec = sleepTimeUsec / 1000000 ;
   ts.tv_nsec = (sleepTimeUsec % 1000000)*1000 ;
   rc = nanosleep( &ts, NULL ) ;
   if ( -1 == rc )
   {
      fprintf( stderr, "nanosleep(%lu,%lu) failed: %d (%s)\n", 
               (unsigned long)ts.tv_sec, (unsigned long)ts.tv_nsec,
               errno, strerror( errno ) ) ;
      return 4 ;
   }

   rc = futimens( fd2, NULL ) ;
   if ( -1 == rc )
   {
      fprintf( stderr, "futimens failed: %d (%s)\n", 
               errno, strerror( errno ) ) ;
      return 5 ;
   }

   close( fd2 ) ;

   return 0 ;
}

Here’s a pair of calls that illustrate the bug.

/vbs/engn/t2> a.out -s 500000 ;  ls --full-time touch*
-r-x--xr-- 1 peeterj pdxdb2 0 2015-02-25 23:16:36.498049000 -0500 touchFirst
-r-x--xr-- 1 peeterj pdxdb2 0 2015-02-25 23:16:36.000000000 -0500 touchSecond
/vbs/engn/t2> a.out -s 500000 ;  ls --full-time touch*
-r-x--xr-- 1 peeterj pdxdb2 0 2015-02-25 23:16:38.900498000 -0500 touchFirst
-r-x--xr-- 1 peeterj pdxdb2 0 2015-02-25 23:16:39.000000000 -0500 touchSecond

The first is the buggy call, and the time goes backwards despite a half second sleep. The second call is okay, because 0.9+0.5 of a second ends up in the next second, so the second “touched” file has a timestamp after the creation of the second file (as expected.)

Thoughts on the “new” C++ style cast operators.

December 24, 2014 C/C++ development and debugging. , , , , , ,

I happen to maintain the DB2 coding standards, which are mostly concerned with portability, and not style. I’ve joked that I was given that job since I had “broken the build” more than anybody else, so was most qualified to let others know how not to do so.

In our coding standards we have a prohibition against the use of exceptions. This is a historical restriction because we’ve built with compilation flags like -qnoeh (no exception handling) on some platforms to get a bit of additional performance. These days the compilers do much better at not degrading performance when exception handling is allowed and not used, but since our performance folks will sell their kids for a 1% improvement, we’ve kept using flags like this and the associated restriction. Components that must (or want) to use exception handling must “firewall” any exceptions, not letting them get thrown to external code (and also explicitly enable exceptions for their code).

We had a note in the coding standards not to use RTTI (Run Time Type Identification), because exceptions are required. That was a confusing and incomplete statement to include in our standards. It was interpreted by one developer as meaning that none of:

dynamic_cast<>()
reinterpret_cast<>()
static_cast<>()
const_cast<>()

were allowed. However, only the dynamic_cast is a RTTI operation, and only the dynamic_cast will throw an exception when the cast doesn’t match the underlying type.

I’ve now fixed up our coding standards. It now references dynamic_cast instead of RTTI.

The DB2 code is still very C’ish, compiled with a C++ compiler. I’d say the bulk of the casts in our code are old style C casts, and that most of our developers (including myself) don’t even know when to use the “new” cast operations. Here’s some thoughts on these:

  • I’ve seen a fair amount of const_cast use in our code, and I know personally how this can be very useful. An example:


volatile int x ;
void foo( int * y ) ;

foo( const_cast(&x) ) ; // compilation error
foo( const_cast(&x) ) ; // allowed.  Just strips the volatile (or const) off the pointer type.
  • A second nice use of const_cast<>() is to enforce type checking in macros. If the macro parameter is supposed to be a pointer to type T, you can enforce that by using a const_cast<T*>. This assumes that you don’t actually care about the const-ness of the pointer, and will force a compile error if the macro is used with any other type.
  • In general I don’t think it’s a bad idea to use the new cast operators, since they represent a hierarchy of weaker than C style casts. You can also search for cast operations by name in a given file if they are used, which is much harder to do with a C style cast.
  • I’m not sure how much use of static_cast<> and reinterpret_cast<> we have in the code.

Declaring my own stupidity, without looking them up, I didn’t personally know how to use the “new style” static_cast and reinterpret_cast operations correctly. I use a C cast by habit unless I’m trying to strip const or volatile attributes (or enforce a type in a macro).

If DB2 coders start using these, it will likely confuse old guys like me for a while, but I figure I can learn about these.

As a step in this direction, I see some helpful looking info in the C++ reference page on explicit_cast

My take on reference page is roughly: If there is a place to use a C cast, then you can use:

  1. const_cast if it compiles. If not you can use:
  2. static_cast. If that doesn’t compile you can use:
  3. reinterpret_cast. If that doesn’t compile you can use:
  4. (In code where exceptions are allowed:) dynamic_cast.
  5. If that doesn’t work or is not allowed you can use:
    A C style cast.