xlC

extern vs const in C++ and C code.

October 5, 2015 C/C++ development and debugging. , , , , , ,

We now build DB2 on linux ppcle with the IBM xlC 13.1.2 compiler. This version of the compiler is a hybrid compared to any previous compilers, retaining the IBM xlC backend for power, but using the clang front end. Because of this we are exposed to a large number of warnings that we don’t see with many other compilers (well we probably do for our MacOSX port, but we do not really have active development on that platform at the moment), and I’ve been trying to take down those counts to manageable levels. Header files that produce warnings have been my first target since they introduce the most repeated noise.

One message that I was seeing hundreds of was

warning: 'extern' variable has an initializer [-Wextern-initializer]

This seemed to be coming from headers that did something like:

#if defined FOO_INITIALIZE_IT_IN_SOME_SOURCE_FILE
extern const TYPE foo[] = { ... } ;
#else
extern const TYPE foo[] ;
#endif


where FOO_INITIALIZE_IT_IN_SOME_SOURCE_FILE is defined at the top of a source file that explicitly includes this header. My attempt to handle the messages was to remove the ‘extern’ from the initialization case, but I was suprised to see link errors as a result of some of those changes. It turns out that there are some subtle differences between different variations of const and extern with an array declaration of this sort. Here’s a bit of sample code:

// t.h
extern const int x[] ;
extern int y[] ;
extern int z[] ;


// t.cc
#if defined WANT_LINK_ERROR
const int x[] = { 42 } ;
#else
extern const int x[] = { 42 } ;
#endif

extern int y[] = { 42 } ;
int z[] = { 42 } ;


When WANT_LINK_ERROR isn’t defined, this produces just one clang warning message

t.cc:8:12: warning: 'extern' variable has an initializer [-Wextern-initializer]
extern int y[] = { 42 } ;
           ^

Note that the ‘extern const’ has no such warning, nor does the non-const symbol that’s been declared ‘extern’ in the header. However, removing the extern from the const case (via -DWANT_LINK_ERROR) results in no symbol ‘x’ available to other consumers. The extern is required for const symbols, but generates a warning for non-const symbols.

It appears that this is also C++ specific. A const symbol in C compiled code is available for external use, regardless of whether extern is used:



$ clang -c t.c
t.c:5:18: warning: 'extern' variable has an initializer [-Wextern-initializer]
extern const int x[] = { 42 } ;
                 ^
t.c:8:12: warning: 'extern' variable has an initializer [-Wextern-initializer]
extern int y[] = { 42 } ;
           ^
2 warnings generated.

$ nm t.o
0000000000000000 R x
0000000000000000 D y
0000000000000004 D z

$ clang -c -DWANT_LINK_ERROR t.c
t.c:8:12: warning: 'extern' variable has an initializer [-Wextern-initializer]
extern int y[] = { 42 } ;
           ^
1 warning generated.
$  nm t.o
0000000000000000 R x
0000000000000000 D y
0000000000000004 D z


whereas that same symbol requires extern if it is const in C++:


$ clang++ -c t.cc
t.cc:8:12: warning: 'extern' variable has an initializer [-Wextern-initializer]
extern int y[] = { 42 } ;
           ^
1 warning generated.
$ nm t.o
0000000000000000 R x
0000000000000000 D y
0000000000000004 D z



$ clang++ -c -DWANT_LINK_ERROR t.cc
t.cc:8:12: warning: 'extern' variable has an initializer [-Wextern-initializer]
extern int y[] = { 42 } ;
           ^
1 warning generated.
$ nm t.o
0000000000000000 D y
0000000000000004 D z


I hadn’t expected the const to interact this way with extern. I am guessing that C++ allows for the compiler to not generate symbols for global scope const variables, unless you ask for that by using extern, whereas with C you get the symbol like-it-or-not. This particular message from the clang front end is only for non-const extern initializations, making across the board fixing of messages for extern initialization of the sort above trickier. This makes it so that you can’t do an across the board replacement of extern in initializers for a given file without first ensuring that the symbol isn’t const. It looks like dealing with this will have to be done much more carefully than I first tried.

Debugging with optimization: -g8

May 8, 2015 C/C++ development and debugging. , , , , ,

It was suggested to me to try -g8 for debugging some optimized code (xlC compiler, linuxppcle platform).  Turns out that there is a whole set of -gN options with the xlC compiler that allow various levels of debug instrumentation.

For the optimization bug that I was debugging, -g8 made the problem go away, but with -g5 I was able to break at the nearest case: block and determine that the data in the variables was good coming into that case statement.  Since what appears to come out of that case statement is bad, this isolates the problem significantly.

decoding some powerpc rotate-mask instructions

November 4, 2014 C/C++ development and debugging. , , , , , , ,

I’m looking at what might be an issue in optimized code, where we are seeing some signs that expected byteswapping is not occuring when optimization is enabled. Trying to find exactly where the code is that does this swapping was been a bit challenging, so I decided to look a simple 2-byte swap sequence in a standalone program first. For such code I see the following in the listing file (xlC compiler)

  182| 00002C rlwinm   5405C23E   1     SRL4      gr5=gr0,8
  182| 000030 rlwinm   5400063E   1     RN4       gr0=gr0,0,0xFF
  182| 000034 rldimi   7805402C   1     RI8       gr5=gr0,8,gr5,0xFFFFFF00

The SRL4, RN4, and RI8 “mnemonics” are internal compiler codes meant to be “intuitive”, but I didn’t find them intuitive until I figured out what the instructions actually did. Here’s a couple reverse engineering notes for that task. Tackling the first rlwinm instruction, note that the raw instruction corresponds to:

0x5405C23E == rlwinm r5,r0,24,8,31

Here’s what the Power ISA says about rlwinm:

Rotate Left Word Immediate then AND with Mask 

rlwinm RA,RS,SH,MB,ME

n  <= SH
r  <= ROTL_32 ((RS)_32:63 , n)
m  <= MASK(MB+32, ME+32)
RA <= r & m

The contents of register RS are rotated 32 left SH bits. A
mask is generated having 1-bits from bit MB+32
through bit ME+32 and 0-bits elsewhere. The rotated
data are ANDed with the generated mask and the
result is placed into register RA.

To interpret this we have to look up the meanings of all the MASK and ROTL operations. My first attempt at that, I got the meaning of MASK() wrong, since I was counting bits from the wrong end. I resorted to the following to figure out the instruction, using gcc inline asm immediate operand constraints “i”, to build up the instruction I wanted to examine the effects of:

#include <stdio.h>

#define rlwinm( output, input, sh, mb, me ) \
   __asm__( "rlwinm %0, %1, %2, %3, %4" \
          : "=r"(output) \
          : "r"(input), "i"(sh), "i"(mb), "i"(me) \
          : )

int main()
{
   long x = 0x1122334455667788L ;
   long y ;

   rlwinm( y, x, 24, 8, 31 ) ;

   printf("0x%016lX -> 0x%016lX\n", x, y ) ;

   return 0 ;
}

This generates an rlwinm instruction with the SH,MB,ME=24,8,31 triplet that I’d found in the listing. This code produces:

0x1122334455667788 -> 0x0000000000556677

Observing the effects of the instruction in this concrete example makes it easier to interpret the effects of the instruction. That seems to be:

   long y = ((int)x << 24) | (char)(x >> 8) ;
   y  |= (y << 32) ; (* The ROTL_32 operation in the ISA appears to have this effect, but it is killed in the mask application *)
   y &= 0xFFFFFF ;

Now the internal mnemonic “SRL4 …,8” has a specific meaning. It looks like it means Shift-Right-Lower4ByteWord 8 bits. It’s intuitive when you know that the L here means Lower. I didn’t guess that, and wondered what the hell shift RightLeft meant.

What does RN4 mean? That instruction was:

0x5400063E == rlwinm r0,r0,0,24,31

This has no shift, but applies a mask, and that mask has 16 bits less ones in it. This appears to be an AND with 0xFF. A little test program using “rlwinm( y, x, 0, 24, 31 )” this time confirms this, as it produces:

0x1122334455667788 -> 0x0000000000000088

What could the R and N have meant? Knowing what the instruction does, I’d now guess RotateNone(andMask).

Finally, how about the RI8 operation? This time we have

0x7805402C == rldimi r5,r0,8,32

The PowerISA says of this:

Rotate Left Doubleword Immediate then Mask Insert  

rldimi RA,RS,SH,MB 

n  <= sh_5 || sh_0:4
r  <= ROTL_64 ((RS), n)
b  <= mb_5 || mb_0:4
m  <= MASK(b, ¬n)
RA  <= r&m | (RA) & ¬ m
The contents of register RS are rotated 64 left SH bits. A
mask is generated having 1-bits from bit MB through bit
63-SH and 0-bits elsewhere. The rotated data are
inserted into register RA under control of the generated
mask.

Let’s also see if an example makes this easier to understand. This time a read/write modifier + is required on the output operand

#include <stdio.h>

#define rldimi( inout, input, sh, mb ) \
   __asm__( "rldimi %0, %1, %2, %3" \
          : "+r"(inout) \
          : "r"(input), "i"(sh), "i"(mb) \
          : )

int main()
{
   long x = 0x1122334455667788L ;
   long y = 0x99aabbccddeeff12L ;
   long yo = y ;

   rldimi( y, x, 8, 32 ) ;

   printf("0x%016lX,0x%016lX -> 0x%016lX\n", x, yo, y ) ;

   return 0 ;
}

This produces:

0x1122334455667788,0x99AABBCCDDEEFF12 -> 0x99AABBCC66778812

It appears that the effect is:

y = (y & ~0xFFFFFF00L) | ((x << 8) & 0xFFFFFF00L) ;

I find it tricky to understand this from the PowerISA description, so if I encountered different values of SH,MB I’d probably run them through this little reverse engineering program. That said, at least the meaning of RI8 in -qlist output is now clear.