LzLabs

What is a non-degree student?

February 17, 2019 Incoherent ramblings , , , ,

In various places on this blog, I’ve mentioned courses that I took as a non-degree student at UofT:

I was recently asked what a non-degree student was, which is a good question, because I think it is a fairly obscure educational path. Here is how UofT describes their non-degree option:

“Non-degree studies is for those with previous university experience who wish to upgrade their university record to qualify for graduate school, a professional program, or for personal interest. Non-degree students enrol in credit courses, for which they have the prerequisites, but are not proceeding towards a degree.”

There are limits of what you can take as a non-degree student. You cannot, for example, take graduate physics courses, nor any courses from engineering. The engineering restriction seems to be because engineering (and computer science, and a few other programs), have a higher price tag. The restriction against taking graduate physics courses as a non-degree student appeared to be arbitrary — I suspect that the grad physics administrator really didn’t want to be bothered, and was happy with the fact that somebody had once imposed that restriction. There also isn’t a large set of people that are clamoring to take grad physics courses just because they are interesting, which makes it easy not to care about removing that restriction.

When I started my non-degree courses, my work at IBM had started to become very routine, and I was seriously questioning my career choices. I’d started off with an interest in the sciences, especially physics, and somehow had ended up as a computer programmer!? At a point of reflection, it is easy to look back and say to your self “how the hell did that happen?” My work at IBM (DB2 LUW) was excellent work from a compensation point of view, and lots of it had been really fun, interesting, and challenging. However, the opportunities to learn on the job were limited, and I was generally feeling under utilized.

I ended up with an unexpected life change event, and took the opportunity to try to reset my career path. IBM offered a flex work program (i.e. 80% pay and hours), and I took used that program to go back to school part time. I ended up taking most of the interesting 4th year grad physics courses, except the two GR courses that I’d still like to take. I had put myself on the path for new employment in a scientific computing field (or perhaps PhD studies down the line. I figured that once I had filled in some of my knowledge gaps, I’d be able to find work that would allow me to both exploit my programming skills, work on a product that mattered, perhaps even learn (science) on the job.

Because I was aiming for scientific computing work, where I figured my 20 years of programming experience would be more relevant than an undergraduate physics degree, non-degree studies was an excellent fit for me. Like any other student in the classes I took, I attended lectures, did the problem sets and exams, and got a grade for each course.

What I didn’t get was any sort of credential for the courses I took. I did end up with 2500 pages of PDF notes for the classes that I took — in my eyes that’s as good as a 2nd degree, but if I did end up looking for that scientific computing work, I’d have to convince my employer of that.

I’m now done with my non-degree studies, and did a followup M.Eng degree so I could take some grad physics courses. This should be the time that I should be looking for that scientific computing work. Why didn’t I switch gears? Well, part way through my M.Eng, I got poached from IBM to work at LzLabs. My work at LzLabs has been way too much fun, and is going to be an awesome addition to our product once completed. A transition from a mega company like IBM to one with ~100 (?) employees wasn’t one that I expected, and perhaps I’ll still end up eventually with scientific computing work, but if that happens it will probably be in the far future. For now, I’m working at LzLabs full time, and not looking back.

I still have a strong affinity for physics, but my plan is to go back to unstructured recreational studies, on my own schedule, once again without any care of credentials.

Mainframe development: a story, chapter 1.

April 19, 2018 Mainframe , , , , , , , , , , , , ,

Once upon a time, in a land far from any modern developers, were languages named COBOL and PL/I, which generated programs that were consumed by a beast known as Mainframe. Developers for those languages compiled and linked their applications huddled around strange luminous green screens and piles of hole filled papers while chanting vaguely latin sounding incantations like “Om-padre-JCL-beget-loadmodule-pee-dee-ess.”

In these ancient times, version control tools like git were not available. There was no notion of makefiles, so compilation and link was a batch process, with no dependency tracking, and no parallelism. Developers used printf-style debugging, logging trace information to files.  In order to keep the uninitiated from talking to the Mainframe, files were called datasets.  In order to use graphical editors, developers had to repeatedly feed their source to the Mainframe using a slave named ftp, while praying that the evil demon EBCDIC-conversion didn’t mangle their work. The next day, they could go back and see if Mainframe accepted their offering.

[TO BE CONTINUED.]

Incidentally, as of a couple days ago, I’ve now been working for LzLabs for 2 years.  My work is not yet released, nor announced, so I can’t discuss it here yet, but it can be summarized as really awesome.  I’m still having lots of fun with my development work, even if I have to talk in languages that the beast understands.

Propublica’s IBM age discrimination investigation

March 22, 2018 Incoherent ramblings , , , , ,

Not too long after I quit IBM for LzLabs in 2016, I was sent a copy of Pro-publica’s survey about age discrimination based firing and forced retirement at IBM. It appears that this survey was just the start of a very long investigation, and they’ve now published their story.

I wasn’t forced out of IBM, and am only ~45 years old, but at the time I had close to 20 years at IBM (including my student internship), and could see the writing on the wall. Technically skilled people with experience were expendable, and being fired or retired with gusto. To me it looked like 25 years at IBM was the firing threshold, unless you took the management path or did a lot of high visibility customer facing work.

IBM’s treatment of employees in the years leading up to when I quit was a major part of my decision to leave. I considered my position at IBM vulnerable for a number of reasons. One was my part time status (80% pay and hours), as I’d been slowly studying physics at UofT with a plan of a future science based job change. Another was that I was a work in the trenches kind of person that did not have the high visibility that looked like it was required for job security in the new IBM where the quarterly firing had gotten so pervasive that you could trip on the shrapnel.

Even after two years I still use “we” talking about my time as an IBMer working on DB2 LUW, as I worked with people that were awesome (some of which I still work with at LzLabs.) Despite now competing with IBM, I hope they stop shooting themselves in the gut by disposing of their skilled employees, and by treating people as rows in resource spreadsheets. It is hard to imagine that this will end well, and it’s too easy to visualize an IBM headstone sharing a plot with HP and Sun.

When I was recruited for LzLabs, my options seemed like continue working for IBM for <= 5 more years before I too got the ax, or to ride into the wild west working as a contractor for a company that was technically still a “startup”. Many startups don’t make it 5 years before folding, so even in the worst case it looked like no bigger risk than IBM, but I thought I was going to have a lot of fun on the ride. LzLabs was just coming out of stealth mode when I was interviewed, but had an astounding ~100 people working at that point! Salaries add up, so it was clear to me that LzLabs was not really a startup in the conventional sense of the word.

It is amusing to read the Pro-publica article now, as most of LzLabs employees are probably over 65. At 45 I’ve been singled out in staff meetings as the “young guy”. Many of the LzLabs employees are technically scary, and know the mainframe cold. I once wrote a simple PowerPC disassembler, but that’s a different game than “disassembling” 390 hex listings by chunking it into various fixed size blocks hex sequences in an editor so it can be “read” by eye!

In less than one month I’ll have been working for LzLabs for 2 years, about six months of which was a contractor before LzLabs Canada was incorporated. Two years ago, if you had mentioned JCL, LE, PL/I, COBOL, QSAM or VSAM (to name a few) to me, I’d have known that seeing COBOL is a good reason to get to an eye wash station pronto (it still is), but would not have even recognized the rest. It’s been fun learning along the way, and I continually impress myself with the parts that I’ve been adding to the LzLabs puzzle. Our technology is amazing and I think that we are going to really kick some butt in the marketplace.

LinkedIn replies to headhunters

September 25, 2017 Incoherent ramblings , , , , , , , , , , ,

I blundered upon my messaging history on LinkedIn the other day, and noticed that it has, for the most part, transitioned from chats with IBMers that I was saying goodbye to (and the ever growing ex-IBMer population that I now know) to rather canned responses to headhunters.

Like contact requests from anybody I don’t remember having worked with, I ignore those from headhunters.  I’ll reply to the headhunter connect requests with a terse “Sorry, I don’t accept requests from people I haven’t worked with personally”.

Going through my replies to the last 10 headhunters who explicitly messaged me, it appears I’m pretty consistent, and most of my replies were fairly close to the following:

Hi XXX,

Thanks for reaching out. I enjoy my current work, which is challenging and interesting, the potential of the company I am working for, and my compensation. I’m not currently interested in a job change.

Peeter

The company that you are recruiting for would have to offer really damn interesting work to get me to defect from LzLabs at the moment.

There were two headhunters that got non-canned responses:

1) A banking and financial sector headhunter got a more direct response:

Hi XXX, thank you for reaching out, but I’m not interested.

This may surprise people, but it’s a moral choice.

I picked up a microeconomics text from the bookshelf of our local Unionville recycling depot (the best priced second hand book store in Markham). That text book was packed with enough Hamilton and Lagrangian equations to make any physicist (or want-to-be physicist like me) at home.  Application of those techniques would surely be interesting, and I was being targeted by a recruiter for a company where that probably would have been possible.  However, it would take a lot more than that chance to make me work directly for a financial parasite.

Yes, I know that I spent 20 years working in the guts of DB2 LUW, which is a product that is used in many financial institutions.  Yes, I know that I am now working for a company with a mainframe solution that is going to be used by many financial institutions.  Both of these cases have a level of indirection that influences my attitude.

If I wasn’t employed, or I had my mortgage paid off, perhaps I’ll feel less hostile to the financial sector.  However, in the near term, I’m certainly not going to work directly for one of the leeches.

2) google.

Hi XXX,

I’m very happy with my current job, which is challenging, interesting, pays very well, and is with a company that has potential I find very appealing.

Google was previously a company that I found intriguing as a possible employer, but has recently demonstrated aspects of authoritarian political correctness that make it much less appealing. There is also evidence of political bias, anti free-speech tendencies, and censorship related to the google products that I find very unsettling given the power and scope of its technology. It is not at all clear that I would be comfortable working at google in its current state.

Peeter

This response was one that surprised me when I wrote it, but I think it is honest.

I would have previously considered working for google if the conditions were right.  When I was at IBM, I never accepted any interview requests from google.  The rationale for that choice was knowledge that relocation was required for any interesting technical work at google (their Toronto lab was marketing only), and I made it clear that relocation was not an option in any correspondence.  I’ve been rebuked by colleagues for that hard line position on relocation, since interviewing at google is said to be really fun.

In recent times, I have been continually reading and hearing of political bias at google.  I’d expect a company that wields so much power to take a non-partisan political position, but they seem to have actively attempted to bias the recent opportunistic-psychopath vs narcissistic-idiot competition in the US, and also appear to be actively attempting to introduce questionable social engineering (biased search rank manipulation, selective demonetization, …) into their products.   In spite of this, even in recent times, had google had google offered up interesting work at interesting compensation levels, without a relocation requirement, perhaps I would have bitten hard enough to interview.

The recent James Damore fiasco is a game changer.  Damore’s primary crime appears to be have been the use of the psychological term neuroticism (a “big five” personality trait that seems to roughly be a measure of negative emotion) without explicit inline definition in his memo. If you are going to fire somebody and make them a scapegoat just to appease the diversity police, then you become uninteresting as an employer.  I just finished working for IBM, which seems to have made it their business to treat people as entries in a HR ledger, irrespective of competence.  It will take some hard sales work to pique interest in google when their HR department is evidently trying to be orders of magnitude more insane than IBMs.  Unless there’s some evidence of HR reform at google, I suspect google technical recruiting is going to get really difficult until their treatment of Damore has been forgotten.

IBM and government are both strong evidence that insanity scales with organization size.  With google clearly growing in size, I am not holding my breath for the chance that it will reverse any of its tendencies down the path toward organizational dementia.

Unpacking a PL/I VSAM keyed write loop.

June 7, 2017 Mainframe , , , , ,

I found myself faced with the task of understanding the effects of a PL/I WRITE loop that does the initial sequential load of a VSAM DATASET.  The block of code I was looking at had the following declarations:

     dcl IXUPD FILE UNBUFFERED KEYED env(vsam);
     dcl
        01 recArea,
            03 recPrefix,
                05 recID        PIC'(4)9' init (0),
                05 recKeyC      CHAR (4)  init (' '),
            03 recordData       CHAR (70) init (' ');

     dcl recIndx FIXED BIN(31) INITIAL(0);

     dcl keyListSize fixed bin(31) initial(10);
     dcl keyList(10) char(8);

As a C++ programmer, there are a few of oddities here:

  • Options for the FILE are specified at the file declaration point (or can be), not at the OPEN point.  They can also be specified at the OPEN point.  The designers of PL/I seem to have been guided by the general principle of “why have one way of doing something, when it can be done in an infinite variety of possible ways”.
  • There is a hybrid “structure & variable” declaration above.  recArea is like an object of an unnamed structure, containing nested parts (with lots of ugly COBOL like nesting specifications to show the depth of the various “structure” members).  It’s something like the following struct declaration (with c++11 default initializer specifiers): 

    #include <stdio.h>

int main() {
    struct {
        struct {
            char recID[4]{'0', '0', '0', '0'};
            char recKeyC[4]{' ', ' ', ' ', ' '};
        } recPrefix;
        char recordData[70]{ ' ', ' ', /* ... 70 spaces total */ };
    } recArea;

    printf( "recID: %.4s\n", recArea.recPrefix.recID );
    printf( "recKeyC: '%.4s'\n", recArea.recPrefix.recKeyC );

    return 0;
}

    To PL/I’s credit, only ~45 years after the creation of PL/1 did C++ add a simple way of encoding default structure member initializers.

    We’ll see below that PL/I lets you access the inner members without any qualification if desired (i.e. recID == recArea.recPrefix.recId). The PL/I compiler writer is basically faced with the unenviable task of continually trying to guess what the programmer could have possibly meant.

  • The int32_t types have the annoying “mainframe”ism of being referred to as 31-bit integers (FIXED BIN(31)). Even if the high bit in pointers is ignored by the hardware (allowing the programmer to set 0x80000000 as a flag, for example for end of list in a list of pointers), that doesn’t mean that the registers aren’t fully 32-bit, nor does it mean that a 32-bit integer isn’t representable. I can’t for the life of me understand why a 32-bit integer variable should be declared as FIXED BINARY(31)?
  • The recID variable is declared with a PICTURE specification, as we also saw in COBOL code. PIC ‘9999’ (or PIC'(4)9′, for “short”), means that the character array will have four (EBCDIC) digits in it. I don’t quite understand this specification in this case, since the code (to follow) seems to put ‘RNNN’, where N is a digit in this field.

Here’s how the declarations above are used:

    
     keyList(1) = 'R001';
     keyList(2) = 'R002';
...
     OPEN FILE(IXUPD) OUTPUT;

     put skip list ('====== Write record to file by key.');
     do while (recIndx &lt; keyListSize);
        recIndx = recIndx + 1;
        recID = recIndx;
        recKeyC = 'Abcd';
        recordData = 'Data for ' || keyList(recIndx);
        write FILE(IXUPD) FROM(recArea) KEYFROM(keyList(recIndx));
     end;
     put skip list (recIndx, ' records is written to file by key.');

     CLOSE FILE(IXUPD);

My guess about what this ‘WRITE FROM(recArea)’ would do is to create records of the form:

0001AbcdData for R001
0002AbcdData for R002
0003AbcdData for R003
...

However, the VSAM DATASET (which was created with key offset 0, and key size 8), actually ends up with:

R001    Data for R001
R002    Data for R002
R003    Data for R003
...

Despite the fact that we are writing from recArea, which includes the recID and recKeyC fields (numeric and character respectively), only the non-key portion of the WRITE “data payload” ends up hitting the disk.

If that is the case, where do the spaces in the key-portion of the records come from? Again, the C programmer in me is interfering with my understanding. I look at:

dcl keyList(10) char(8);
keyList(1) = 'R001';

and think that keyList(1) = “R001\x00\x00\x00\x00”, but it must actually be space filled in PL/I! This seems to be confirmed emperically, based on the expected results for the test, but I can also see it in the debugger after manually relocating the 32-bit mainframe address:

(gdb) p keyLen
$1 = 8
(gdb) p /x aKey + 0x7ffbc4000000
$2 = 0x7ffbc5005740
(gdb) set target-charset EBCDIC-US
(gdb) p (char *)$2
$3 = 0x7ffbc5005740 "R001    R002    R003    R004    R005    R006    R007    R008    R009    R010    "

The final form of the records in the VSAM DATASET (mainframe for a file), is now fully understood. Note that the data disagrees with the PICTURE specification for the recID field in the recData variable declaration, but that’s okay, at least for this part of the program, since there is never any store to that field that is non-numeric. Would anything even have to have been written to recID or recKeyC … I suspect not? Once we have R00N in that part of the record what happens if we read it into recData with the numeric only PICTURE specification? Does that raise a PL/1 condition?

ps. Notice how the payload for the keyList array entries is nicely packed into memory. This is done in a very non-C like fashion with no requirement for an array of pointers and the corresponding cache hit loss those pointers create when accessing a big multilevel C array.