## The C compiler is too forgiving! sizeof(variable_name+1) allowed?

April 28, 2022 C/C++ development and debugging. , , , ,

I carelessly passed:

sizeof(s.st_size+1)


to an allocator call, instead of:

s.st_size+1


and corrupted memory nicely.

What the hell would sizeof(variable+1) even mean, and why on earth would the compiler think that is anything close to valid? Both gcc and clang, each with -Wall, are completely quiet about this error!

## Debugging a C coding error from an XPLINK assembly listing.

There are at least two$${}^1$$ z/OS C calling conventions, the traditional “LE” OSLINK calling convention, and the newer$${}^2$$ XPLINK convention.  In the LE calling convention, parameters aren’t passed in registers, but in an array pointed to by R1.  Here’s an example of an OSLINK call to strtof():

*  float strtof(const char *nptr, char **endptr);
LA       r0,ep(,r13,408)
LA       r2,buf(,r13,280)
LA       r4,#wtemp_1(,r13,416)
L        r15,=V(STRTOF)(,r3,4)
LA       r1,#MX_TEMP3(,r13,224)
ST       r4,#MX_TEMP3(,r13,224)
ST       r2,#MX_TEMP3(,r13,228)
ST       r0,#MX_TEMP3(,r13,232)
BASR     r14,r15
LD       f0,#wtemp_1(,r13,416)


R1 is pointed to r13 + 224 (a location on the stack). If the original call was:

float f = strtof( mystring, &err );

The compiler has internally translated it into something of the form:

STRTOF( &f, mystring, &err );

where all of {&f, mystring, &err} are stuffed into the memory starting at the 224(R13) location. Afterwards the value has to be loaded from memory into a floating point register (F0) so that it can be used.  Compare this to a Linux strtof() call:

* char * e = 0;
* float x = strtof( "1.0", &e );
400b74:       mov    $0x400ef8,%edi ; first param is address of "1.0" 400b79: movq$0x0,0x8(%rsp)       ; e = 0;
400b82:       lea    0x8(%rsp),%rsi       ; second param is &e
400b87:       callq  400810 <strtof@plt>  ; call the function, returning a value in %xmm0


Here the input parameters are RDI, RSI, and the output is XMM0. Nice and simple. Since XPLINK was designed for C code, we expect it to be more sensible. Let’s see what an XPLINK call looks like. Here’s a call to fmodf:

*      float r = fmodf( 10.0f, 3.0f );
LD       f0,+CONSTANT_AREA(,r9,184)
LD       f2,+CONSTANT_AREA(,r9,192)
L        r6,=A(__fmodf)(,r7,76)
L        r5,=A(__fmodf)(,r7,72)
BASR     r7,r6
NOP      9
LDR      f2,f0
STE      f2,r(,r4,2144)
*
*      printf( "fmodf: %g\n", (double)r );


There are some curious details that would have to be explored to understand the code above (why f0, f2, and not f0,f1?), however, the short story is that all the input and output values in (floating point) registers.

The mystery that led me to looking at this was a malfunctioning call to strtof:

*      float x = strtof( "1.0q", &e );
LA       r2,e(,r4,2144)
L        r6,=A(strtof)(,r7,4)
L        r5,=A(strtof)(,r7,0)
LA       r1,+CONSTANT_AREA(,r9,20)
BASR     r7,r6
NOP      17
LR       r0,r3
CEFR     f2,r0
STE      f2,x(,r4,2148)
*
*      printf( "strtof: v: %g\n", x );


The CEFR instruction converts an integer to a (hfp32) floating point representation, so we appear to have strtof returning it’s value in R3, which is an integer register. That then gets copied into R0, and finally into F2 (and after that into a stack spill location before the printf call.) I scratched my head about this code for quite a while, trying to figure out if the compiler had some mysterious way to make this work that I wasn’t figuring out. Eventually, I clued in. I’m so used to using a C++ compiler that I forgot about the old style implicit int return for an unprototyped function. But I had included <stdlib.h> in this code, so strtof should have been prototyped? However, the Language Runtime reference specifies that on z/OS you need an additional define to have strtof visible:

#define _ISOC99_SOURCE
#include <stdlib.h>


Without the additional define, the call to strtof() is as if it was prototyped as:

int strtof( const char *, char ** );


My expectation is that with such a correction, the call to strtof() should return it’s value in f0, just like fmodf() does. The result should also not be garbage!

Footnotes:

1.  There is also a “metal” compiler and probably a different calling convention to go with that.  I don’t know how metal differs from XPLINK.
2. Newer in the lifetime of the mainframe means circa 2001, which is bleeding edge given the speed that mainframe development moves.

## Playing with c++11 and posix regular expression libraries

I was curious how the c++11 std::regex interface compared to the C posix regular expression library. The c++11 interfaces are almost as easy to use as perl. Suppose we have some space separated fields that we wish to manipulate, showing an order switch and the original:

my @strings = ( "hi bye", "hello world", "why now", "one two" ) ;

foreach ( @strings )
{
s/(\S+)\s+(\S+)/'$&' -> '$2 $1'/ ; print "$_\n" ;
}


The C++ equivalent is

   const char * strings[] { "hi bye", "hello world", "why now", "one two" } ;

std::regex re( R"((\S+)\s+(\S+))" ) ;

for ( auto s : strings )
{
std::cout << regex_replace( s, re, "'$&' -> '$2 $1'\n" ) ; }  We have one additional step with the C++ code, compiling the regular expression. Precompilation of perl regular expressions is also possible, but that is usually just as performance optimization. The posix equivalent requires precompilation too void posixre_error( regex_t * pRe, int rc ) { char buf[ 128 ] ; regerror( rc, pRe, buf, sizeof(buf) ) ; fprintf( stderr, "regerror: %s\n", buf ) ; exit( 1 ) ; } void posixre_compile( regex_t * pRe, const char * expression ) { int rc = regcomp( pRe, expression, REG_EXTENDED ) ; if ( rc ) { posixre_error( pRe, rc ) ; } }  but the transform requires more work: void posixre_transform( regex_t * pRe, const char * input ) { constexpr size_t N{3} ; regmatch_t m[N] {} ; int rc = regexec( pRe, input, N, m, 0 ) ; if ( rc && (rc != REG_NOMATCH) ) { posixre_error( pRe, rc ) ; } if ( !rc ) { printf( "'%s' -> ", input ) ; int len ; len = m[2].rm_eo - m[2].rm_so ; printf( "'%.*s ", len, &input[ m[2].rm_so ] ) ; len = m[1].rm_eo - m[1].rm_so ; printf( "%.*s'\n", len, &input[ m[1].rm_so ] ) ; } }  To get at the capture expressions we have to pass an array of regmatch_t’s. The first element of that array is the entire match expression, and then we get the captures after that. The awkward thing to deal with is that the regmatch_t is a structure containing the start end end offset within the string. If we want more granular info from the c++ matcher, it can also provide an array of capture info. We can also get info about whether or not the match worked, something we can do in perl easily my @strings = ( "hi bye", "helloworld", "why now", "onetwo" ) ; foreach ( @strings ) { if ( s/(\S+)\s+(\S+)/$2 $1/ ) { print "$_\n" ;
}
}


This only prints the transformed line if there was a match success. To do this in C++ we can use regex_match

const char * pattern = R"((\S+)\s+(\S+))" ;

std::regex re( pattern ) ;

for ( auto s : strings )
{
std::cmatch m ;

if ( regex_match( s, m, re ) )
{
std::cout << m[2] << ' ' << m[1] << '\n' ;
}
}


Note that we don’t have to mess around with offsets as was required with the Posix C interface, and also don’t have to worry about the size of the capture match array, since that is handled under the covers. It’s not too hard to do wrap the posix C APIs in a C++ wrapper that makes it about as easy to use as the C++ regex code, but unless you are constrained to using pre-C++11 code and can also live with a Unix only restriction. There are also portability issues with the posix APIs. For example, the perl-style regular expressions like:

   R"((\S+)(\s+)(\S+))" ) ;


work fine with the Linux regex API, but that appears to be an exception. To make code using that regex work on Mac, I had to use strict posix syntax

   R"(([^[:space:]]+)([[:space:]]+)([^[:space:]]+))"


Actually using the Posix C interface, with a portability constraint that avoids the Linux regex extensions, would be horrendous.

## More C++11 notes from reading Stroustrup: nothrow, try, inline & unnamed namespace, initialized new

Here’s more notes from reading Stroustrup’s “The C++ Programming Language, 4th edition”

## throw() as noexcept equivalent

throw() without any exception types can be used as an equivalent to the new noexcept keyword. Stroustrup also mentions that explicit throw() clauses

void foo() throw( e1, e2 ) ;


haven’t worked out well in practise, and is deprecated.

## try scopes as function body

It turns out that try clauses can be used as function bodies, as in

void foo( void )
try {
}
catch ( ... )
{
}


This can also be done for constructor and destructor bodies as in

X::X( T1 v, T2 w )
try{
: f1( v )
, f2( w )
}
catch ( ... )
{
}


so that a throw in the class field member construction can also be caught.

## Inline (default) namespace

There is a mechanism for namespace versioning. Suppose that you want a new V2 namespace to be the default, you can do:

namespace myproject
{
inline namespace V2
{
struct X {
int x ;
int y ;
} ;
void foo( const X & ) ;
}

namespace V1
{
struct X {
int x ;
} ;

void foo( const X & ) ;
}
}


Existing callers of the library that are using V1 interfaces can continue to work unmodified, but new callers will use the V2::X and V2::foo interfaces, and the library can provide both interfaces, one for compatibility and another for new code:

void myproject::V2::foo( const myproject::V2::X & )
{
// ...
}

void myproject::V1::foo( const myproject::V1::X & )
{
// ...
}


## Unnamed namespaces.

I’d once seen unnamed namespaces as a modern C++ (more general) replacement for static functions. To see if such namespace functions are optimized away in the same fashion as a static function, I tried

#include <stdio.h>

namespace
{
void foo()
{
printf( "ns:foo\n" ) ;
}
}

int main()
{
foo() ;

return 0 ;
}


This example uses printf and not std::cout because I wanted to look at the assembly listing and cout’s listing, at least on a mac, was completely abysmal. foo() was optimized away, but that’s a lot easier to see in the C printf listing:

$make c++ -o n -std=c++11 -O2 n.cc$ otool -tV n | less
n:
(__TEXT,__text) section
_main:
0000000100000f70        pushq   %rbp
0000000100000f71        movq    %rsp, %rbp
0000000100000f74        leaq    0x2b(%rip), %rdi        ## literal pool for: "ns:foo"
0000000100000f7b        callq   0x100000f84             ## symbol stub for: _puts
0000000100000f80        xorl    %eax, %eax
0000000100000f82        popq    %rbp
0000000100000f83        retq


## at_quick_exit

There’s now also a mechanism to exit and avoid global destructors and atexit routines from being evaluated. Here’s an example

#include <cstdlib>
#include <iostream>

extern "C"
void normalexit()
{
std::cout << "normalexit\n" ;
}

extern "C"
void quickCexit()
{
std::cout << "quickCexit\n" ;
}

void quickCPPexit()
{
std::cout << "quickCPPexit\n" ;
}

class X
{
public:
~X()
{
std::cout << "X::~X()\n" ;
}
} x ;

int main( int argc, char ** argv )
{
atexit( normalexit ) ;
std::at_quick_exit( quickCexit ) ;
std::at_quick_exit( quickCPPexit ) ;

if ( argc == 1 )
{
std::quick_exit( 3 ) ;
}

when run without arguments (argc == 1), we get

$./at quickCPPexit quickCexit  whereas if the normal exit processing is allowed to complete we see global destructors and regular atexit calls $ ./at 1
normalexit
X::~X()


Observe, unlike atexit, which can only (portably) take extern “C” defined functions, at_quick_exit can take functions with both C and C++ linkage.

## Enum default

It was not obvious to me what the default value for an enum class (or enum) should be (the first value, an invalid value, zero, …)? It turns out that the default is zero, as printed by the following fragment

#include <iostream>

enum class x { v = 1, w } ;
enum y { vv = 1, ww } ;

int main()
{
x e1 = {} ;
y e2 = {} ;
std::cout << (int)e1 << '\n' ;
std::cout << e2 << '\n' ;

return 0 ;
}


Note that an explicit cast is required for enum class values, but not for enum, which are by default, int convertible.

## default initialization with new

The uniform initializer syntax can also be used with new calls. Here’s an example with uninitialized and default initialized double allocations

#include <stdio.h>

int main()
{
double * d1 = new double ;
double * d2 = new double{} ;

printf( "%g %g\n", *d1, *d2 ) ;

return 0 ;
}


Observe that we get nice garbage values for *d1, but *d2 is always 0.0:

$./d -1.49167e-154 0$ ./d
0 0
$./d 1.72723e-77 0$ ./d
-2.68156e+154 0


## initializer_list

I remember really wanting a feature like this eons ago when I first wrote a matrix template class in 1st year. Here’s a sample of how it could be used

#include <iostream>
#include <vector>
#include <string>

template <unsigned r, unsigned c>
class m
{
std::vector<double> mat ;

public:

m() : mat(r*c) {}

m( std::initializer_list<double> i ) : mat( r * c )
{
if ( i.size() > ( r * c ) )
{
}

int p{} ;
for ( auto v : i )
{
mat[ p++ ] = v ;
}
}

void dump( const std::string & n ) const
{
const char * sep = ": " ;
std::cout << n ;

for ( auto v : mat )
{
std::cout << sep << v ;
sep = ", " ;
}

std::cout << '\n' ;
}
} ;

int main()
{
m< 3, 2 > v1 ;
m< 3, 2 > v2{ 0., 1., 2., 3., 4. } ;

v1.dump( "v1" ) ;
v2.dump( "v2" ) ;

m< 3, 2 > v3{ 0., 1., 2., 3., 4., 5., 6., 7. } ;

return 0 ;
}


This produces the two dumps and the expected std::terminate call for the wrong (too many) parameters on the third construction attempt

\$ ./i
v1: 0, 0, 0, 0, 0, 0
v2: 0, 1, 2, 3, 4, 0
libc++abi.dylib: terminating with uncaught exception of type m<3u, 2u>::bad_init
Abort trap: 6


## Notes for Stroustrup’s “The C++ Programming Language, 4th Ed.”: nothrow new, noexcept, noreturn, static cons, initializer_list

I recently purchased Stroustrup’s C++11 book [1], after borrowing it a number of times from the Markham public library (it’s very popular, and only offered for short term loan) . Here are some notes of some bits and pieces that were new to me for this round of reading.

## nothrow new

In DB2 we used to have to compile with -fcheck-new or similar, because we had lots of code that predated new throwing on error (c++98). There is a form of new that explicitly doesn’t throw:

void * operator new( size_t sz, const nothrow_t &) noexcept ;


I don’t know if this was introduced in c++11. If this was a c++98 addition, then it should be used in almost all the codebases new calls. When I left DB2 there were still some platform compilers (i.e. AIX xlC which doesn’t use the clang front end like linuxppcle64 xlC) that were not c++11 capable, so if this explicit nothrow isn’t c++98, it probably can’t be used.

## Unnamed function parameters

It is common to see function prototypes without named parameters, such as

void foo( int, int ) ;


I did not realize that is also possible in the function definition, as in code like the following where a parameter has been dropped or left as a placeholder for future use

void foo( int x, int )
{
printf( "%d\n", x ) ;
}


Not naming the parameter is probably a good way to get rid of unused parameter warnings.

This is very likely not a c++11 addition. I just didn’t realize the language allowed for it, and had never seen it done.

## No return attribute

Looks like __attribute__ extensions are being baked right into the language, as in

[[noreturn]] void exit( int ) ;


I wonder if this is also in the plan for C?

C++11 explicitly requires static variable constructors are initialized using a “call-once” mechanism

class x
{
public:
x() ;
} ;

void foo( void )
{
static x v() ;
}


Here there is no data race if foo() is executed concurrently in a number of threads. I remember seeing DB2 code that did this (and opening a defect to have it “fixed”), since I had no idea if it would work. We didn’t (and couldn’t yet) use -std=c++11, so it’s anybody’s guess what that does without that option and on older pre c++11 compilers.

## Implied type initializer lists.

In a previous post I mentioned the c++11 uniform initialization syntax, but the basic idea is that is instead of

int x(1) ;
int y(0) ;


or

int x = 1 ;
int y = 0 ;


c++11 now allows

int x{1} ;
int y{} ;


Here the variables are initialized with values 1, and 0 (the default). The motivation for this was to provide an initializer syntax that could be used with container classes. Here’s another variation on the initializer list initialization

int x = int{} ;
int y = int{3} ;


which can be reduced to

int x = {} ;
int y = {3} ;


where the types of the lists are implied. I don’t see much value add to use this equals-list syntax in the examples above. Where this might be useful is in templated code to provide defaults

template <typename T>
void foo( T x, T v = {} ) ;


## Runtime values for default arguments.

I don’t know if this is new to C++11, but the book points out that default arguments can be runtime determined values. Initially, my thought on this was that it is good that is not well known, since it would be confusing. I did however, come up with a scenerio where this could be useful. I wrote some code like the following the other day

extern bool g ;

inline int foo( )
{
int res = 0 ;

if ( g )
{
// first option
}
else
{
// second option
}

return res ;
}


The global g was precomputed at the shared library startup point (effectively const without being marked so). My unit test of this code modified the value of g, which was a hack and I admit ugly. It looked like

BOOST_AUTO_TEST_CASE( basicTest )
{
for ( auto b : {false, true} )
{
g = b ;

int res = foo() ;

BOOST_REQUIRE( res >= 0 ) ;
}
}


This has a side effect of potentially changing the global. A different way to do this would have been

extern bool g ;

inline int foo( bool internalOverrideOfGlobalForTesting = g )
{
int res = 0 ;

if ( internalOverrideOfGlobalForTesting )
{
// first option
}
else
{
// second option
}

return res ;
}


The test code could then be rewritten as

BOOST_AUTO_TEST_CASE( basicTest )
{
for ( auto b : {false, true} )
{
int res = foo( b ) ;

BOOST_REQUIRE( res >= 0 ) ;
}
}


This doesn’t touch the global (an internal value), but still would have allowed for testing both codepaths. The fact that this “feature” exists may not actually be in this case, since my interface was a C interface. Does a

## noexcept

Functions that intend to provide a C interface can use the noexcept keyword. That allows the compiler to enforce the fact that such functions should provide a firewall that doesn’t let any exceptions through. Example:

// foo.h
#if defined __cplusplus
#define EXTERNC extern "C"
#define NOEXCEPT noexcept
#else
#define EXTERNC
#define NOEXCEPT
#endif

EXTERNC void foo(void) NOEXCEPT ;

// foo.cc
#include "foo.h"
int foo( void ) NOEXCEPT
{
int rc = 0 ;
try {
//
}
catch ( ... )
{
// handle error
rc = 1 ;
}

return rc ;
}


If foo does not catch all exceptions, then the use of noexcept will drive std::terminate(), like a throw from a destructor does on some platforms.

# References

[1] Bjarne Stroustrup. The C++ Programming Language, 4th Edition. Addison-Wesley, 2014.