C/C++ development and debugging.

splitting the last git commit into two

April 23, 2020 C/C++ development and debugging. , ,

In the blog post, Split a commit in two with Git, Emmanuel provides a super clear explanation of how to split an old commit into multiple commits, separating that commit into different commits, each with a subset of the files initially committed.

It took me a while before I could figure out how to apply this to the very last commit.  Here’s the required git magic:

git log -n 1 > m
git reset HEAD^
git add ...
git commit -m "First part"
git add ...
git commit -m "Second part"

The differences are really to just skip the first and last rebase steps (don’t do an interactive rebase, and don’t continue that rebase when done.) This was probably obvious to the author of the more general instructions.

Note that before resetting HEAD to the previous commit, I collect the current commit message, under the assumption that portions of it will be used in either of the two (or more) new commit messages.  If you don’t do that, you can fish it out of your history by looking at ‘git reflog’ to see what the message was before mucking around with HEAD.

Interesting z/OS (clang based) compiler release notes.

December 13, 2019 C/C++ development and debugging. , , , ,

The release notes for the latest z/OS C/C++ compiler are interesting.  When I was at IBM they were working on “clangtana”, a clang frontend melded with the legacy TOBY backend.  This really surprised me, but was consistent with the fact that the IBM compiler guys kept saying that they were continually losing their internal funding — that project was a clever way to do more with less resources.  I think they’d made the clangtana switch for zLinux by the time I left, with AIX to follow once they had resolved some ABI incompatibility issues.  At the time, I didn’t know (nor care) about the status of that project on z/OS.

Well, years later, it looks like they’ve now switched to a clang based compiler frontend on z/OS too.  This major change appears to have a number of side effects that I can imagine will be undesirable to existing mainframe customers:

  • Compiler now requires POSIX(ON) and Unix System Services.  No more compilation using JCL.
  • Compiler support for 31-bit applications appears to be dropped (64-bit only!)
  • Support for C, FASTLINK, and OS linkage conventions has been dropped (XPLINK only.)
  • Only ibm-1047 is supported for both source and runtime character set encoding.
  • C89 support appears to have been dropped.
  • Hex floating support has been dropped.
  • No decimal floating point support.
  • SIMD support isn’t implemented.
  • Metal C support has been dropped.

i.e. if you want C++14, you have to be willing to give up a lot to get it.  They must be using an older clang, because this “new” compiler doesn’t include C++17 support.  I’m surprised that they didn’t even manage multiple character set support for this first compiler release.

It is interesting that they’ve also dropped IPA and PDF support, and that the optimization options have changed.  Does that mean that they’ve actually not only dropped the old Montana frontend, but also gutted the whole backend, switching to clang exclusively?

An example of Linux/glibc symbol versioning.

September 20, 2019 C/C++ development and debugging. , , , , , , ,

Here’s a little introduction to Linux/glibc symbol versioning.  The sources, linker version scripts, and makefile from this post can be found here in github.

The definitive reference for this topic is Ulrich Drepper’s dsohowto document.

Linux symbol versioning is a very logical way to construct a definitive API for your product.  If you are careful, you can have a single version of your shared library that is binary compatible with all previous versions of code that have linked it, and additionally hide all symbols that you don’t want your API consumer to be able to use (as if all non-exported symbols have a “static” like scope within the library itself).

That visibility hiding can be done in other ways (such as using static, or special compiler options), but utilizing a symbol version linker script to do so also has all the additional binary compatibility related benefits.

Suppose that we have a “v1.1” API with the following implementation:

#include <stdio.h>

void internal_function_is_not_visible()
{
   printf( "You can't call me directly.\n" );
}

void foo( int x )
{
   internal_function_is_not_visible();

   printf( "foo@@V1: %d\n", x );
}

and we build this into a shared library like so:

$ make libfoo.1.so
rm -f libfoo.1.so
cc foo.c -g -Wl,--version-script=symver.map -fpic -o libfoo.1.so -Wl,-soname,libfoo.so -shared
rm -f libfoo.so
ln -s libfoo.1.so libfoo.so

Everything here is standard for building a shared library except for the –version-script option that is passed into the linker with -Wl,. That version script file has the following contents:

$ cat symver.map
MYSTUFF_1.1 {
	  global:
foo;
	  local: *;
	};

This defines a “V1.1” API where all the symbols that are exported with a symbol version @@MYSTUFF_1.1. Note that internal_function_is_not_visible is not in that list. It’s covered in the local: catch-all portion of the symbol version file. Code that calls foo does not look out of the ordinary:

void foo(int);

int main()
{
   foo(1);

   return 0;
}

Compiling and linking that code is also business as usual:

$ make caller1
cc caller.c -g -Wl,-rpath,`pwd` -o caller1 -lfoo -L.

However, look at the foo symbol reference that we have for this program:

$ nm caller1 | grep foo
                 U foo@@MYSTUFF_1.1

If we run this, we get:

$ ./caller1
You can't call me directly.
foo@@V1: 1

If you add in a call to internal_function_is_not_visible() you’ll see that compilation fails:

void foo(int);
void internal_function_is_not_visible();

int main()
{
   foo(1);
   internal_function_is_not_visible();

   return 0;
}

$ make caller1
cc caller.c -g -Wl,-rpath,`pwd` -o caller1 -lfoo -L.
/run/user/1002/ccqEPYcu.o: In function `main':
/home/pjoot/symbolversioning/caller.c:7: undefined reference to `internal_function_is_not_visible'
collect2: error: ld returned 1 exit status
make: *** [caller1] Error 1

This is because internal_function_is_not_visible is not a visible symbol. Cool. We now have versioned symbols and symbol hiding. Suppose that we now want to introduce a new binary incompatible change too our foo API, but want all existing binaries to still work unchanged. We can do so by introducing a symbol alias, and implementations for both the new and the OLD API.

#include <stdio.h>

void internal_function_is_not_visible()
{
   printf( "You can't call me directly.\n" );
}

void foo2( int x, int y )
{
   if ( y < 2 )
   {
      internal_function_is_not_visible();
   }
   printf( "foo@@V2: %d %d\n", x, y );
}

void foo1( int x )
{
   internal_function_is_not_visible();
   printf( "foo@@V1: %d\n", x );
}

This is all standard C up to this point, but we now add in a little bit of platform specific assembler directives (using gcc specific compiler sneaks) :

#define V_1_2 "MYSTUFF_1.2"
#define V_1_1 "MYSTUFF_1.1"

#define SYMVER( s ) \
    __asm__(".symver " s )

SYMVER( "foo1,foo@" V_1_1 );
SYMVER( "foo2,foo@@" V_1_2 );

We’ve added a symbol versioning alias for foo@@MYSTUFF_1.2 and foo@MYSTUFF_1.1. The @@ one means that it applies to new code, whereas the @MYSTUFF_1.1 is a load only function, and no new code can use that symbol. In the symbol version script we now introduce a new version stanza:

$ cat symver.2.map
MYSTUFF_1.2 {
	  global:
foo;
};
MYSTUFF_1.1 {
	  global:
foo;
	  local: *;
};

$ make libfoo.2.so
rm -f libfoo.2.so
cc foo.2.c -g -Wl,--version-script=symver.2.map -fpic -o libfoo.2.so -Wl,-soname,libfoo.so -shared
rm -f libfoo.so
ln -s libfoo.2.so libfoo.so

If we call the new V1.2 API program, like so:

void foo(int, int);
int main()
{
   foo(1, 2);
   return 0;
}

our output is now:

$ ./caller2
foo@@V2: 1 2

Our “binary incompatible changes” are two fold. We don’t call internal_function_is_not_visible if our new parameter is >= 2, and we print a different message.

If you look at the symbols that are referenced by the new binary, you’ll see that it now explicitly has a v1.2 dependency:

$ nm caller2 | grep foo
                 U foo@@MYSTUFF_1.2

The lldb TUI (text user interface)

August 26, 2019 C/C++ development and debugging. , , ,

It turns out, like GDB, that lldb has a TUI mode too, but it’s really simplistic.  You enter with

(lldb) gui

at which point you get a full screen of code or assembly, and options for register exploration, thread and stack exploration, and a variable view.  The startup screen looks like:

If you tab over to the Threads window, you can space select the process, and drill into the stack traces for any of the running threads

You can also expand the regsiters by register class:

I’d like to know how to resize the various windows.  If you resize the terminal, the size of the stack view pane seems to remain fixed, so the symbol names always end up truncated.

Apparently this code hasn’t been maintained or developed since it was added.  Because there is no console pane, you have to set all desired breakpoints and continue, then pop into the GUI to look at stuff, and then <F1> to get back to the console prompt.  It’s nice that it gives you a larger view of the code, but given that lldb already displays context around each line, the lldb TUI isn’t that much of a value add in that respect.

This “GUI” would actually be fairly usable if it just had a console pane.

gdb pretty print of structures

March 9, 2017 C/C++ development and debugging. , , , ,

Here’s a nice little gdb trick for displaying structure contents in a less compact format

(gdb) set print pretty on
(gdb) p dd[0]
$4 = {
  jfcb = {
    datasetName = "PJOOT.NVS1", ' ' <repeats 34 times>,
    .
    .
    .
    vols = {"<AAAiW", "\000\000\000\000\000", "\000\000\000\000\000", "\000\000\000\000\000", "\000\000\000\000\000"},
  },
  block_size = 800,
  device_class = 32 '\040',
  device_type = 15 '\017',
  disp_normal = 8 '\010',
  disp_cond = 8 '\010',
  volsers = 0x7fb71801ecd6 "<AAAiW",
} 

compare this to the dense default

(gdb) set print pretty on
(gdb) p dd[0]
$5 = {jfcb = {datasetName = "PJOOT.NVS1", ... vols = {"<AAAiW", "\000\000\000\000\000", "\000\000\000\000\000", "\000\000\000\000\000", "\000\000\000\000\000"}, block_size = 800, block_size_limit = 0, device_class = 32 '\040', device_type = 15 '\017', disp_normal = 8 '\010', disp_cond = 8 '\010', volsers = 0x7fb71801ecd6 "<AAAiW"}

For really big structures (this one actually is, but I’ve pruned a bunch of stuff), this makes the structure print display a whole lot more readable. Additionally, if you combine this with ‘(gdb) set logging on’, then with pretty print enabled you can prune the output by line easily to see just what you want.