Month: May 2016

Marks are in for Winter 2016

May 18, 2016 Incoherent ramblings , , , , , , , , , , ,

I started my formal re-education program back in 2010 after 20 years out of school. The first few courses were particularly tough after such a long break from school (and exam based courses still are), but I’m getting the hang of playing that game again. Here’s my score so far:

Crs Code  Title                                    Wgt  Mrk  Grd    CrsAvg
PHY356H1  Quantum Mechanics I                      0.50  78  B+     C+
PHY450H1  Rel Electrodynamics                      0.50  78  B+     *
PHY456H1  Quantum Mechanics II                     0.50  72  B-     C+
PHY454H1  Continuum Mech                           0.50  85  A      B
PHY485H1  Adv Classical Optics                     0.50  85  A      *
PHY452H1  Basic Stat Mechanics                     0.50  81  A-     B-
PHY487H1  Condensed Matter I                       0.50  80  A-     B+
ECE1254H  Modeling of Multiphysics Systems         0.50      A+
ECE1229H  Advanced Antenna Theory                  0.50      A-
PHY1520H  Quantum Mechanics                        0.50      A-
PHY1610H  Scientific Computing for Physicists      0.50      A+     

This last grad course is the only one of which they gave (informally through email) a non-letter grade (97). That one happened to be very well suited to me, and did not have anything based on exams nor on presentations (just assignments). They were demanding assignments (and fun), so I had to work really hard for that 97.

As a returning student I really suck at classes that have marks that are highly biased towards exams. My days of showing up late for class, sleeping through big chunks of the parts that I did get their in time for, and still breezing through the exams are long gone. Somehow in my youth I could do that, and still be able to quickly and easily barf out all the correct exam answers without thinking about it. I got a 99 in first year calculus doing exactly that, although it helped that the Central Technical School’s math department kicked butt, and left Prof Smith with only a review role.

Now I take a lot more time thinking things through, and also take a lot of time writing up my notes (which would sometimes have been spent better doing practise problems). It’s funny thinking back to undergrad where I had such scorn for anybody that took notes. Now I do just that, but in latex. I would be the object of my own scorn X10, if I met my teenage self again!

First build break at the new job: C++ uniform initialization

May 12, 2016 C/C++ development and debugging. , , , , ,

Development builds at LZ are done with clang-3.8, but there is an alternate nightly build done with the older RHEL7 GCC-4.8.3 compiler (gcc is up to 6.1 now, so the RHEL7 default is truly _ancient_). This bit of code didn’t compile with gcc:

   template <typename mutex_type>
   class shared_lock
   {  
      mutex_type &      m_mutex ;

   public:

      /** construct and acquire the mutex in shared mode */
      explicit shared_lock( mutex_type & mutex )
         : m_mutex{ mutex }
      {  

The error is:

error: invalid initialization of non-const reference of type ‘lz::shared_mutex&’ from an rvalue of type ‘<brace-enclosed initializer list>’

This seems like a compiler bug to me, one that I’d seen when doing my scinet scientific computing course, which mandated the use of at least -std=c++11. In the scinet assignments, I fixed all such issues by using -std=c++14, which worked fine, but I was using gcc-5.3 for those assignments.

It appears that this is a compiler bug, and not just an issue with the c++11 language specification, as I initially thought while doing my scinet assignments. If I rebuild this code with g++-6.1, explicitly specifying -std=c++11 (GCC 6.1 defaults to c++14), then the issue goes away, so specification of -std=c++14 is not required to allow uniform initialization to work in this situation.

Because of being forced to use the older compiler, it looks like I have to fix this by using pre-c++11 syntax:

      explicit shared_lock( mutex_type & mutex )
         : m_mutex( mutex )

My conclusion is that gcc-4.8.3 is not truly up to the job of building c++11 compliant code. I’ll have to be more careful with the language features that I use in the future.

Leaving IBM: A causal analysis.

May 3, 2016 Incoherent ramblings , , , , , , , ,

EDIT: fixed the title; casual -> causal. I am clearly not ever going to be headhunted by any editing or proofreading companies.

I was not looking for a job change, but one found me. There has been persistent headhunter (Google, Microsoft, Facebook, Amazon, …) interest over the years, but any interesting job available also required relocation. I also wanted something compatible with my part time studies, as I have been working 80% part time at IBM to accommodate those studies.

I was contacted by the founder of a new startup. As a long time salaried IBM employee, I am surprised that the possible instability of a startup operation was of interest, but they made a convincing case for the future success of their product ideas and company. Their salary offer was also significantly higher than what I make at IBM, which sure didn’t hurt.

With interesting work as a prospect at this company, an attractive salary, no relocation required, and none of the negatives of current IBM HR practices, I took this offer after a week of deliberation.

Pros of working in DB2

One week of deliberation, especially when I wasn’t looking for a change, is a short amount of time to decide to say goodbye to a company that I’ve worked for about 20 years, which is almost half my life! Both my kids were born while working for IBM, and are now almost grown. I have a roots with the people and work that I am leaving behind.

I did a lot of really fun work in my years with DB2:

– Implemented multithread support and associated reentrancy retrofit of DB2’s Unix client code.
– DB2 Linux porting work, including portability reengineering.
– DB2 64-bit port. This was a project of massive scope. We can now eat terrabytes of RAM for breakfast.
– Development and maintenance of DB2’s internal mutex and atomic implementations, and associated memory ordering mechanisms.
– Lots of fun parts of our platform stackwalk and related post mortem factilities.
– AIX/TPCC performance and exploitation liason.
– DB2 contact for xlC and other compiler coordination.
– Lock-free reimplementation of DB2’s reader-writer mutex. The performance of our original reader-writer mutex code sucked for a number of reasons (including use of an mutex within the mutex.) This was a from scratch implementation where we used a single atomic to track the reader and writer linked lists (indirectly), the reader count, the writer held bit, and a writer reserved bit. This code has stood the test of time and remains one of my proudest creations.
– Implementation of DB2’s asynchronous IO abstraction layer. It’s hard to believe that it wasn’t that long ago that we did all our IO synchronously. This bit of code hides an impressive amount of operating system centric code from our high level development consumers, while squeezing maximum performance from each system.
– Development and maintenance of many other aspects of DB2’s operating system abstraction layer.
– Lead of project branch integration team during internal transition from the CMVC version control system to clearcase.
– Ad-hoc build tooling and makefile maintenance as required.
– Technical owner of DB2’s coding standards
– DB2 pureScale project (distributed shared disk database): Implemented various duplexing, failover and reconstruct aspects of the communications between the DB2 engine and the shared buffer pool and lock manager component.
– Lots of other stuff along the way that took me into various components of DB2.

I started straight from school in ’97 with a low salary (somewhere around $50K CAD). My salary and band rating both progressed very rapidly from there. There were also frequent bonuses in those early days. I was clearly perceived as being of value.

I also really enjoyed the people that I worked with in IBM. I’ve worked with so many very competent and skilled software developers over the years.

So, the pros include:
– Lots of fun and challenging work.
– What seemed like a decent salary.
– Almost nobody else to work for in the Toronto region for whom I could do systems and low level programming.
– Part time options for my educational project and extra time off with the kids.
– Awesome fellow developers. It is humbling and rewarding to work with so many really smart people.
– Working for a startup carries the risk of complete failure.

So why leave?

Many of the reasons to leave DB2 mirror the reasons to stay:

– Fun and challenging work.
– Better salary.
– Flexibility offered for part time work if desired.
– Opportunity to continue to work without relocation, and without having to do something boring like web programming, phone app development, java coding, …
– A collection of really smart developers.
– Staying at IBM can pigeonhole me and leave me viewed as a one trick pony.
– Expectation of a stock distribution and/or bonuses with success.
– The chance to work for a small company, shaping things instead of being a nameless drone.
– No inhuman IBM corporate HR policies to have to observe or be impacted by.

Fun and challenging work.

Now that I have started with LzLabs, I am starting to get a glimpse at just how aggressive and visionary this project is. The scope of it is very impressive, and I’m going to have a lot of fun working on it. My work is likely going to be some combination of concurrency, porting, and build & test infrastructure. These are all things that I am comfortable with and enjoy working on.

Better salary

I was very satisfied with the rate that I achieved my maximum full time equivalent salary at IBM.

I was not satisfied with the way that it stagnated after that. I had not had a raise in a long time. Living in Canada it seems that food costs have 2x’ed in recent years, as have gas prices, hydro bills, and many others. My IBM salary was clearly not even tracking inflation.

The offer I got from LZ certainly made up for that stagnation.

That said, I probably could have asked for a lot more. I got heavily berated by an ex-IBM buddy for accepting the offer I got from LZ, which he said was way too low. He had the good fortune to have been canned by IBM. He’s since moved around and found just how low IBM Canada pays in comparison to others, and is now making $200K USD on the west coast. While my new LZ salary is significantly better than my IBM salary, he said that I shouldn’t have accepted anything less than $240K USD given my skills, and would have easily gotten that, even without requiring relocation. This was surprising to hear. I was clearly pretty clueless about the going rates for the sort of work that I’ve been doing.

My buddy recommends that I get away from LZ in short order (i.e. 6 months) if it shows any signs of not becoming a superstar player. It sounds like I could definitely profit by such a move, but I’ve never been primarily motivated by money. I’d rather play this new game long term and see where it goes.

IBM Canada capitalizes on the sort of salary comparison cluelessness that I had. It also capitalizes on what used to be a monopoly on systems and low level programming work. Without anybody else local to work for they have been able to keep salaries low.

With the ease of working remotely becoming so pervasive, IBM won’t be able to play this game as effectively anymore. This is clearly evident by the mass migrations that seem to be occurring (not even counting the frequent IBM purges).

One trick pony.

Loyalty to an employer used to be considered a virtue. I am not sure that is the case anymore. I left on my own terms, but should I have been fired from IBM, I think that the fact that I worked in only one place for 20 years would have hurt, not helped. Leaving IBM has a positive optics impact on my resume for future work.

Part time options.

I’ve started full time with LZ. I’ve been offered the chance to go back to part time in the fall when school starts again. The LZ founder has said of this

“You are going to have so much fun that you won’t have any desire to continue with an attempt to become a failed physicist. We have lots of those working for us already.”

Although this sounds like it is condescending, he is proud of his collection of very smart people, including all his “failed physicists”, which he said in a way that it sounded like a complement to their intelligence.

One of the LZ hiring strategies is to ask new hires “who is the best/smartest developer that you have ever worked with” (I can’t play my role in that game for a year since I was an IBM band 9). Judging by the interactions that I have had so far at the company, I can confirm that I am in with a really good group.

Inhuman IBM HR

I’ve mentioned that I really enjoyed the people that I worked with in DB2 and IBM. There are so many truly on the ball experienced developers in the pool. It was a real loss to leave IBM and not be able to continue working with these people anymore.

However, in recent years, I have seen IBM treat many of those skilled and experienced developers as disposable. Again and again and again, the most experienced developers are tossed like trash when the firing purges occur. These are in many cases guru level developers with irreplaceable knowledge. Some call this agism, but it may be more accurate to call it salarism, because I think their true crime was getting paid too much in the view of HR that is eying three new hire recent grads for the same total salary (or perhaps three employees in an offshore IBM lab where salaries are uniformly less).

As a 20 year employee, I had only 5 years before I also hit the apparent disposability threshold. Given that, the risks of a joining a startup that might fail are severely minimized. Worst case, if the start up fails, then I am left looking for a job, which is probably also an inevitability if staying on at IBM. The only real loss in that case is the IBM severance that I walk away from by leaving voluntarily, and the new larger salary should compensate for that, provided I sack some of it away for a rainy day.

When I did a final walk around of the lab, saying bye to so many people I had worked with for so long (some that I had never even met in person!), I observed many that had reached or surpassed what I thought of as the 25 year purge threshold criteria. There is no apparent sane rationale for who gets to stay and who gets the knife. That uncertainty for those who remain must be very hard to deal with. Some who I talked to did not even have this perception of disposability, which was interesting to observe.

Tossing employees who have expired is not the only example of cutthroat IBM HR I have seen. In one case, a co-worker who had become the goto guy for a complex component of our code had the bad fortune of getting a promotion too close to one of the firing purges. There is an internal ranking scheme within IBM, and in recent years, anybody with a rank three was toast when the purges ran. Unfortunately, a promotion usually means you are dropped temporarily in rank, since you have to compete with more experienced developers once you have your new band rating. In this case, the promotion in question, reducing the employee rank from 2 to 3, which put him into the automatic firing bucket. It seems to me completely insane to spend five years training somebody, and then essentially fire him for a promotion.

I have also observed two cases where IBMers were fired within what seemed like months of returning to work after having recovered from cancer. That’s a callous action that demonstrates the people with their fingers on the triggers truly don’t think of employees as people. They are resources. Just are just numbers to be shuffled in spreadsheets. Observe that I refuse to use the cowardly term “Resource Action” (or worse RA, or RAed) for firing that IBM HR employs. I am guessing that this a term invented so that HR employees feel less sadistic when they have to run a spreadsheet computation to see who gets the knife. I won’t miss that sort of corporate sadism, and it was a strong factor associated with my choice to leave IBM.

Weighing all the factors.

Looking at all the pros and cons, it seemed clear to me that joining LZ and leaving IBM was in my best interest.

It was certainly a scary move. I don’t even have a salary or real job offer since LZ is not yet incorporated in Canada, and I am currently working as a contractor. Now that I have started, things are no longer quite so scary.

This was a huge change, and has happened really fast. All said, I am happy to have been able to leave on my own terms, and am going to have fun playing a new game at LZ.

Would I work for IBM again in the future?

I had trouble with the binary “Would you work for IBM again” question that was part of leaving IBM feedback form. Having enjoyed my work and the people I worked with over these years, my first instinct was to answer yes, and I did so. However, corporate IBM would have to reform the way it treats its people significantly to be attractive again. I suspect IBM is too large for that to ever occur.

Notes on C++11 and C++14 from scientific computing for physicists

May 1, 2016 C/C++ development and debugging. , , , , , , , , , , , , , , , , , , , , , , , ,

I recently wrapped up all the programming assignments for PHY1610, Scientific Computing for Physicists

In all the assignments, we were required to compile with either

-std=c++11

or

-std=c++14

It’s possible to use those options and still program using the older C++98 syntax, but I also used this as an opportunity to learn some new style C++.

With the cavaet that we were provided with boilerplate code for a number of assignments, there was a non-trivial amount of code written for this course:

$ cloc `cat f` 2>&1 | tee o
     186 text files.
     177 unique files.                                          
       4 files ignored.

http://cloc.sourceforge.net v 1.60  T=0.88 s (197.6 files/s, 16868.5 lines/s)
-------------------------------------------------------------------------------
Language                     files          blank        comment           code
-------------------------------------------------------------------------------
C++                            111           1710           1159           7317
C/C++ Header                    62            819           1525           2237
-------------------------------------------------------------------------------
SUM:                           173           2529           2684           9554
-------------------------------------------------------------------------------

A lot of this code involved calling into external libraries (fftw3, cblas, lapack, gsl, netcdf, MPI, silo, boost exceptions, boost unittest, …) and was pretty fun to write.

Looking through my submissions, here are some of the newer language features that ended up in my code. Keep in mind that new for me is relative to the C++ language features that I was able to use in DB2 code, which is restricted by the features made available by the very oldest compiler we were using accross all platform offerings.

Using statements

I had only seen using statements for namespace selection, as in

using namespace std ;

This is, however, a more general construct, and also allows for what is effectively a scope limited typedef with a more natural syntax. Example:

using carray = rarray<std::complex<double>, 1> ;

Compare this to

typedef rarray<std::complex<double>, 1> carray ;

With the using syntax, the beginner programmer’s issue of remembering the order for the type,typename pair in a typedef statement is obliterated.

I got quite used to using using by the end of the course.

Testing language levels

The following macros were helpful when experimenting with different language levels:

#if defined __cplusplus && (__cplusplus >= 201103L)
   #define HAVE_CPLUSPLUS_11
#endif

#if defined __cplusplus && (__cplusplus >= 201402L)
   #define HAVE_CPLUSPLUS_14
#endif

enum class

C++11 introduces an ‘enum class’, different from an enum. For example, instead of writing:

/**
   interval and derivative solver methods supplied by gsl
 */
enum solver
{
   bisection,
   falsepos,
   brent,
   newton,
   secant,
   steffenson
} ;

you would write:

/**
   interval and derivative solver methods supplied by gsl
 */
enum class solver
{
   bisection,
   falsepos,
   brent,
   newton,
   secant,
   steffenson
} ;

The benefit of this compared to the non-class enum is that the enumeration names are not in the global scope. You would write

void foo( const solver s ) 
{
   if ( s == solver::falsepos )
}

not

void foo( const solver s ) 
{
   if ( s == falsepos )
}

This nicely avoids namespace clashes.

That is not the only benefit to C++11 enums. C++11 enums can also be forward referenced, provided the storage class of the enum is also specified.

If you have ever worked on code that is massively coupled and interdependent (such as DB2), you have seen places where piles of headers have to get dragged in for enum bodies, because it is not possible to forward reference an enum portably. This is a very nice feature!

A simple example of a forward declared C++11 enum is:

enum solver : int ;
void foo( const solver s ) ;

enum solver : int
{
  x = 0, y = 1
} ;

Or, using the non-global enum class syntax:

enum class what : int ;
void foo( const what s ) ;

enum class what : int
{
  x = 0, y = 1
} ;

I didn’t actually use enum classes for enum forward referencing in my phy1610 assignments, because they were too simple to require that.

There is huge potential for using enums with storage classes in DB2 code. I expect that is also true for many other huge scale C++ codebases. The fact that this feature does not have appear to be tied to a requirement to also use ‘enum class’ is very nice for transforming legacy code. I left IBM before the day of seeing the use of compilers that allowed that on all platforms, but can imagine there will be some huge potential build time savings once C++11 compilers are uniformly available for DB2 code (and the code is ported to compile with C++11 enabled on all platforms).

As a side note, the storage class qualification, even if not being used for forward referencing is quite nice. I used it for return codes from main, which have to fit within one byte (i.e. within the waitpid waitstatus byte). For example:

enum class RETURNCODES : unsigned char
{
    SUCCESS       ///< exit code for successful exectution
   ,HELP          ///< exit code when -help (or bad option is supplied)
   ,PARSE_ERROR   ///< exit code if there's a parse error */
   ,EXCEPTION     ///< exit code if there's an unexpected exception thrown */
} ;

Uniform initialization

A new initialization paradigm is available in C++11. Instead of using constructor syntax for initialization, as in

/**
   Input parameters for gsl solver iteration.
 */
struct iterationParameters
{
   const Uint     m_max_iter ;  ///< Maximum number of iterations before giving up.
   const double   m_abserr ;    ///< the absolute error criteria for convergence.
   const double   m_relerr ;    ///< the relative error criteria for convergence.
   const bool     m_verbose ;   ///< verbose output

   iterationParameters( const Uint     max_iter,
                        const double   abserr,
                        const double   relerr,
                        const bool     verbose ) :
         m_max_iter(max_iter),
         m_abserr(abserr),
         m_relerr(relerr),
         m_verbose(verbose)
   {
   }
} ;

one could write

/**
   Input parameters for gsl solver iteration.
 */
struct iterationParameters
{
   const Uint     m_max_iter ;  ///< Maximum number of iterations before giving up.
   const double   m_abserr ;    ///< the absolute error criteria for convergence.
   const double   m_relerr ;    ///< the relative error criteria for convergence.
   const bool     m_verbose ;   ///< verbose output

   iterationParameters( const Uint     max_iter,
                        const double   abserr,
                        const double   relerr,
                        const bool     verbose ) :
         m_max_iter{max_iter},
         m_abserr{abserr},
         m_relerr{relerr},
         m_verbose{verbose}
   {
   }
} ;

This is a little foreign looking and it is easy to wonder what the advantage is. One of the advantages is that this syntax can be used for container initialization. For example, instead of

std::vector<int> v ;
v.push_back( 1 ) ;
v.push_back( 2 ) ;
v.push_back( 3 ) ;

you can just do

std::vector<int> v{ 1, 2, 3 } ;

This is called uniform initialization, since this mechanism was extended to basic types as well. For example, instead of initializing an array with an assignment operator, as in

   constexpr struct option long_options[] = {
     { "help",   0, NULL, 'h' },
     { "number", 1, NULL, 'n' },
     { "lower",  1, NULL, 'l' },
     { "upper",  1, NULL, 'u' },
     { NULL,     0, NULL, 0   }
   } ;

you can write

   constexpr struct option long_options[]{
     { "help",   0, NULL, 'h' },
     { "number", 1, NULL, 'n' },
     { "lower",  1, NULL, 'l' },
     { "upper",  1, NULL, 'u' },
     { NULL,     0, NULL, 0   }
   } ;

Instead of just providing a special mechanism to initialize container class objects, the language was extended to provide a new initialization syntax that could be used to initialize contain those objects and all others.

However, this is not just a different syntax for initialization, because there the types have to match strictly. For example this init of a couple stack variables will not compile

   int more{3} ;
   float x1{-2.0} ;
   size_t size{meta.numThreads*20} ;

What is required is one of

   float x1{-2.0f} ;

   // or

   double x1{-2.0} ;

Additionally, suppose that meta.numThreads has int type. Such a uniform initialization attempt will not compile, since the product is not of type size_t. That line can be written as:

   size_t size{(size_t)meta.numThreads*20} ;

   // or:
   size_t size = meta.numThreads*20 ;

I found uniform initialization hard on the eyes because it looked so foreign, but did eventually get used to it, with one exception. It seems to me that a longer initialization expression like the following is harder to read

double x{ midpoint( x1, x1 + intervalWidth ) } ;

than

double x = midpoint( x1, x1 + intervalWidth ) ;

There were also cases with -std=c++11 where uniform init and auto variables (see below) did not interact well, producing errors later when my auto-uniform-init’ed variables got interpreted as initializer lists instead of the types I desired. All such errors seemed to go away with -std=c++14, which seemed to generally provide a more stable language environment.

New string to integer functions

The c++11 standard library has new string to integer functions
http://en.cppreference.com/w/cpp/string/basic_string/stoul
which are more convenient than the strtoul functions. These throw exceptions on error, but still allow the
collection of errno and error position if you want them.

using Uint = std::uintptr_t ;

/**
   Register sized signed integer type for loop counters and so forth.
 */
using Sint = std::intptr_t ;

/**
   wrapper for stoul to match the type of Uint above.
 */
#if defined _WIN64
   #define strToUint std::stoull
#else
   #define strToUint std::stoul
#endif

There are other similar functions like std::stod, for string to double conversion. There were also opposite convertors, such as to_string, for converting integer types to strings. For example:

const std::string filename{ fileBaseName + "_" + std::to_string( rank ) + ".out" } ;

Static assertions.

DB2 had a static assertion implementation (OSS_CTASSERT, or sqlzStaticAssert?) but there is now one in the standard. Here’s an example using the Uint “typedef” above:

/**
   Force a compilation error if size assumptions are invalid.
 */
inline void strToUintAssumptions()
{
#if defined _WIN64
   static_assert( sizeof(Uint) == sizeof(unsigned long long), "bad assumptions about sizeof uintptr_t, long long" ) ;
#else
   static_assert( sizeof(Uint) == sizeof(unsigned long), "bad assumptions about sizeof uintptr_t, long" ) ;
#endif
}

The advantage of static_assert over a typedef (variable sized array) implementation like DB2 HAD is that compilers likely produce a better error message when it fails (instead of something unintuitive like “reference of array location at offset -1 is invalid”).

Boost exceptions.

While not part of c++11, the boost exception classes were available for my assignments. These are pretty easy to use. As setup you define some helper classes, which really just provide a name for the exception, and a name to identify any of the data that you’d like to throw along with the underlying exception. This could look like the following for example:

#include <boost/exception/exception.hpp>
#include <boost/exception/info.hpp>

struct error : virtual std::exception, virtual boost::exception { } ;
struct regex_match_error : virtual error { } ;

struct tag_match_input ;
typedef boost::error_info<tag_match_input,std::string> match_info ;

struct tag_match_re ;
typedef boost::error_info<tag_match_re,std::string> re_info ;

struct tag_intdata ;
typedef boost::error_info<tag_intdata,long> intdata_info ;

Such classes would be best in a namespace since they are generic, but I didn’t bother for all these assignments.

I used the boost exceptions for a couple things. One of which, of course, was throwing exceptions, but the other was as an assert-with-data backend:

#define ASSERT_NO_ERROR (static_cast<void>(0))
#ifdef NDEBUG
   #define ASSERT_DATA_INT( expr, v1 )          ASSERT_NO_ERROR
   #define ASSERT_DATA_INT_INT( expr, v1, v2 )  ASSERT_NO_ERROR
#else
   #define ASSERT_DATA_INT( expr, v1 )          \
      ( (expr)                                  \
      ? ASSERT_NO_ERROR                         \
      : BOOST_THROW_EXCEPTION(                  \
            assert_error()                      \
               << intdata_info( v1 ) ) )
//...
#endif

This allowed me to assert with data as in

ASSERT_DATA_INT( sz > 0, sz ) ;
ASSERT_DATA_INT_INT( taskNumber < numTasks, taskNumber, numTasks ) ;

This way I get not just the abort from the assert, but also the underlying reason, and can dump those to the console with no additional effort than catching any other boost exception:

//...
#include <boost/exception/diagnostic_information.hpp>

int main( int argc, char ** argv )
{
   try {
      auto expected{7} ;

      ASSERT_DATA_INT_INT( argc == expected, argc, expected ) ;
   }
   catch ( boost::exception & e )
   {
      auto s { boost::diagnostic_information( e ) } ;
      std::cout << s << std::endl ;
      // ...

This generates something like:

$ ./bassert
bassert.cc(11): Throw in function int main(int, char**)
Dynamic exception type: boost::exception_detail::clone_impl<assert_error>
std::exception::what: std::exception
[tag_intdata*] = 1
[tag_intdata2*] = 7

I wonder how efficient constructing such an exception object is? When pre-processed the assertion above expands to

      ( (argc == expected) ? (static_cast<void>(0)) :
     ::boost::exception_detail::throw_exception_(
     assert_error() << intdata_info( argc ) << intdata2_info( expected )
     ,__PRETTY_FUNCTION__,"bassert.cc",11)
     ) ;

Stepping through this in the debugger I see some interesting stuff, but it included heap (i.e. new) allocations. This means that this sort of Boost exception may malfunction very badly in out of memory conditions where it is conceivable that one would want to throw an exception.

The runtime cost can’t be that inexpensive either (when the assert is triggered). I see four function calls even before the throw is processed:

assert_error const& boost::exception_detail::set_info(assert_error const&, boost::error_info const&)-0x4
assert_error const& boost::exception_detail::set_info(assert_error const&, boost::error_info const&)-0x4
assert_error::assert_error(assert_error const&)-0x4
void boost::throw_exception(assert_error const&)-0x4

and the total instruction count goes up to ~140 from 4 for the NDEBUG case (with optimization). Only 5 instructions get executed in the happy codepath. This is what we want in exception handling code: very cheap when it’s not triggered, with all the expense moved to the unhappy codepath.

The negative side effect of this sort of error handling looks like a lot of instruction cache bloat.

Boost test

The boost test library is also not a C++11 feature, but new for me, and learned in this course. Here’s a fragment of how it is used

#define BOOST_TEST_MAIN
#define BOOST_TEST_MODULE test

#define BOOST_TEST_DYN_LINK

#include <boost/test/unit_test.hpp>
#include <vector>

BOOST_AUTO_TEST_CASE( testExample )
{
   std::vector<int> v(3) ;

   BOOST_REQUIRE( 3 == v.size() ) ;
   BOOST_REQUIRE_MESSAGE( 3 == v.size(), "size: " + std::to_string( v.size() ) ) ;
}

A boost test after being run looks like:

$ ./test --report_level=detailed --log_level=all
Running 1 test case...
Entering test module "test"
test.cc:9: Entering test case "testExample"
test.cc:13: info: check 3 == v.size() has passed
test.cc:14: info: check 'size: 3' has passed
test.cc:9: Leaving test case "testExample"; testing time: 87us
Leaving test module "test"; testing time: 103us

Test module "test" has passed with:
  1 test case out of 1 passed
  2 assertions out of 2 passed

  Test case "testExample" has passed with:
    2 assertions out of 2 passed

Range for and auto type

The range for is much like perl’s foreach. For example, in perl you could write

my @a = ( 1, 2, 3 ) ;
foreach my $v ( @a )
{
   foo( $v ) ;
}

An equivalent C++ loop like this can be as simple as

std::vector<int> a{1, 2, 3 } ;
for ( auto v : a )
{
   foo( v ) ;
}

You can also declare the list of items to iterate over inline, as in

using iocfg = iohandler::cfg ;
for ( auto c : { iocfg::graphics, iocfg::ascii, iocfg::netcdf, iocfg::noop } )
{
   // ...
}

Observe that, just like perl, C++ no longer requires any explicit type for the loop variable, as it is deduced when auto is specified. It is still strongly typed, but you can write code that doesn’t explicitly depend on that type. I see lots of benefits to this, as you can have additional freedom to change type definitions and not have to adjust everything that uses it.

I can imagine that it could potentially get confusing if all variables in a function get declared auto, but did not find that to be the case for any of the code I produced in these assignments.

One gotcha with auto that I did hit was that care is required in computed expressions. I’d used auto in one case and the result got stored as a large unsigned value, instead of signed as desired (i.e. negative values got stored in unsigned auto variables). In that case I used an explicit type. Extensive use of auto may end up requiring more unit and other test if the types picked are not those that are desired.

std::chrono (ticks.h)

This is a nice portability layer for fine grain time measurements, allowing you to avoid platform specific functions like gettimeofday, and also avoid any composition of the seconds/subseconds data that many such interfaces provide.

Here’s a fragment of a class that allows interval time measurements and subsequent conversion:

class ticks
{
   using clock      = std::chrono::high_resolution_clock ;

   clock::time_point m_sample ;
public:

   static inline ticks sample()
   {
      ticks t ;
      t.m_sample = clock::now() ;

      return t ;
   }

   using duration   = decltype( m_sample - m_sample ) ;

   friend duration operator -( const ticks & a, const ticks & b ) ;
} ;

inline ticks::duration operator -( const ticks & a, const ticks & b )
{
   return a.m_sample - b.m_sample ;
}

inline auto durationToMicroseconds( const ticks::duration & diff )
{
   return std::chrono::duration_cast<std::chrono::microseconds>( diff ).count() ;
}

Note that the last function is using c++14 return type deduction. That does not work without coersion
in c++11, requiring:

inline auto durationToMicroseconds( const ticks::duration & diff )
-> decltype(std::chrono::duration_cast<std::chrono::microseconds>( diff ).count())
{
   return std::chrono::duration_cast<std::chrono::microseconds>( diff ).count() ;
}

which is very ugly.

Random numbers

/**
   A random number generator that produces integer uniformly
   distributed in the interval:

   [a, a + delta N]

   with separation delta between values returned.
 */
template <int a, int delta, int N>
class RandomIntegers
{
   std::random_device                        m_rd ;A
   //std::default_random_engine                m_engine ;
   std::mt19937                              m_engine ;
   std::uniform_int_distribution<unsigned>   m_uniform ;

public:
   /** constuct a uniform random number generator for the specified range */
   RandomIntegers( )
      : m_rd()
      , m_engine( m_rd() )
      , m_uniform( 0, N )
   {
      static_assert( N > 0, "Integer N > 0 expected" ) ;
      static_assert( delta > 0, "Integer delta > 0 expected" ) ;
   }

   /**
      return a uniform random number sample from {a, a + delta, ..., a + delta N}
    */
   int sample()
   {
      auto p = m_uniform( m_engine ) ;

      return a + p * delta ;
   }
} ;

constexpr

Instead of using #defines, one can use completely typed declarations, but still constant using the constexpr keyword. An example

constexpr size_t N{3} ;
std::tuple<int, N> t ;

nullptr

The days of not knowing what header defines NULL and dealing with conflicting definitions are over. Instead of using NULL, we now have a builtin language construct nullptr available.

Lambdas and sort

Custom sorting is really simple in c++ now. Here’s an example of a partial sort (sorting the top N elements, and leaving the rest unspecified). The sort function no longer has to be a function call, and can be specified inline

auto second_greater = [](auto & left, auto & right) { return left.second > right.second ; } ;
std::partial_sort( cvec.begin(),
                   cvec.begin() + N,
                   cvec.end(),
                   second_greater ) ;

The “inline” sort function here is using c++14 lambda syntax. For c++11, the parameter types can’t be auto, so something such as the following might be required

auto second_greater = [](const results_pair & left, const results_pair & right) { return left.second > right.second ; } ;

Useful standard helper methods

The standard library has lots of useful utility functions. I’m sure I only scratched the surface discovering some of those. Some I used were:

std::swap( m_sz, other.m_sz ) ;
std::fill( m_storage.begin(), m_storage.end(), v ) ;
std::copy( b.m_storage.begin(), b.m_storage.end(), m_storage.begin() ) ;
r.first  = std::max( l, m_myFirstGlobalElementIndex ) ;
r.second = std::min( u, m_myLastGlobalElementIndex ) ;

I also liked the copysign function, allowing easy access to the sign bit of a float or double without messing around with extracting the bit, or explicit predicates:

inline double signof( const double v )
{
   return std::copysign( 1.0, v ) ;
}

Mean and standard deviation were also really easy to calculate. Here’s an example that used a lambda function to calculate the difference from the mean to get at the squared difference from the mean:


      m_sum = std::accumulate( v.begin(), v.end(), 0.0 ) ;
      m_mean = m_sum / v.size() ;
      double mean = m_mean ; // for lambda capture

      std::vector<double> diff( v.size() ) ;

      std::transform( v.begin(), v.end(), diff.begin(), [mean](double x) { return x - mean; } ) ;

      m_sq_sum = std::inner_product( diff.begin(), diff.end(), diff.begin(), 0.0 ) ;

decltype

Attempting to mix auto with g++’s ‘-Wall -Werror’ causes some trouble. For example, this doesn’t work

void foo ( const size_t size )
{
   for ( auto i{0} ; i < size ; i++ )
   {
      // ...
   }
}

This doesn’t compile since the i < size portion generates sign vs unsigned comparison warnings. There are a few ways to fix this.

   // specify the type explicitly:
   for ( size_t i{0} ; i < size ; i++ )

   // let the compiler use the type of the size variable:
   for ( decltype(size) i{0} ; i < size ; i++ )

The decltype method is probably of more use in template code. For non-template code, I found that explicitly specifying the type was more readable.

std::valarray (myrarray.h)

The standard library has a vectored array construct, but I was disappointed with the quality of the generated code that I observed. It also turned out to be faster not to use it. For example:

void SineCosineVecOps( std::valarray<float> & s, std::valarray<float> & c, const std::valarray<float> & v )
{
   s = std::sin( v ) ;
   c = std::cos( v ) ;
}

void SineCosineManOps( std::valarray<float> & s, std::valarray<float> & c, const std::valarray<float> & v )
{
   for ( Uint i{0} ; i < ASIZE ; i++ )
   {  
      float theta = v[i] ;

      s[i] = std::sin( theta ) ;
      c[i] = std::cos( theta ) ;
   }
}

when run on a 300 element array executed close to 1.5x slower using the valarray vector assignment operation, and had close to 3x times the instructions (with optimization)!

Perhaps other compilers do better with valarray. g++ 5.3 is certainly not worth using with that container type.