VSAM

Unpacking a PL/I VSAM keyed write loop.

June 7, 2017 Mainframe , , , , ,

I found myself faced with the task of understanding the effects of a PL/I WRITE loop that does the initial sequential load of a VSAM DATASET.  The block of code I was looking at had the following declarations:

     dcl IXUPD FILE UNBUFFERED KEYED env(vsam);
     dcl
        01 recArea,
            03 recPrefix,
                05 recID        PIC'(4)9' init (0),
                05 recKeyC      CHAR (4)  init (' '),
            03 recordData       CHAR (70) init (' ');

     dcl recIndx FIXED BIN(31) INITIAL(0);

     dcl keyListSize fixed bin(31) initial(10);
     dcl keyList(10) char(8);

As a C++ programmer, there are a few of oddities here:

  • Options for the FILE are specified at the file declaration point (or can be), not at the OPEN point.  They can also be specified at the OPEN point.  The designers of PL/I seem to have been guided by the general principle of “why have one way of doing something, when it can be done in an infinite variety of possible ways”.
  • There is a hybrid “structure & variable” declaration above.  recArea is like an object of an unnamed structure, containing nested parts (with lots of ugly COBOL like nesting specifications to show the depth of the various “structure” members).  It’s something like the following struct declaration (with c++11 default initializer specifiers):

    #include <stdio.h>
    
    int main() {
        struct {
            struct {
                char recID[4]{'0', '0', '0', '0'};
                char recKeyC[4]{' ', ' ', ' ', ' '};
            } recPrefix;
            char recordData[70]{ ' ', ' ', /* ... 70 spaces total */ };
        } recArea;
    
        printf( "recID: %.4s\n", recArea.recPrefix.recID );
        printf( "recKeyC: '%.4s'\n", recArea.recPrefix.recKeyC );
    
        return 0;
    }
    

    To PL/I’s credit, only ~45 years after the creation of PL/1 did C++ add a simple way of encoding default structure member initializers.

    We’ll see below that PL/I lets you access the inner members without any qualification if desired (i.e. recID == recArea.recPrefix.recId). The PL/I compiler writer is basically faced with the unenviable task of continually trying to guess what the programmer could have possibly meant.

  • The int32_t types have the annoying “mainframe”ism of being referred to as 31-bit integers (FIXED BIN(31)). Even if the high bit in pointers is ignored by the hardware (allowing the programmer to set 0x80000000 as a flag, for example for end of list in a list of pointers), that doesn’t mean that the registers aren’t fully 32-bit, nor does it mean that a 32-bit integer isn’t representable. I can’t for the life of me understand why a 32-bit integer variable should be declared as FIXED BINARY(31)?
  • The recID variable is declared with a PICTURE specification, as we also saw in COBOL code. PIC ‘9999’ (or PIC'(4)9′, for “short”), means that the character array will have four (EBCDIC) digits in it. I don’t quite understand this specification in this case, since the code (to follow) seems to put ‘RNNN’, where N is a digit in this field.

Here’s how the declarations above are used:

    
     keyList(1) = 'R001';
     keyList(2) = 'R002';
...
     OPEN FILE(IXUPD) OUTPUT;

     put skip list ('====== Write record to file by key.');
     do while (recIndx &lt; keyListSize);
        recIndx = recIndx + 1;
        recID = recIndx;
        recKeyC = 'Abcd';
        recordData = 'Data for ' || keyList(recIndx);
        write FILE(IXUPD) FROM(recArea) KEYFROM(keyList(recIndx));
     end;
     put skip list (recIndx, ' records is written to file by key.');

     CLOSE FILE(IXUPD);

My guess about what this ‘WRITE FROM(recArea)’ would do is to create records of the form:

0001AbcdData for R001
0002AbcdData for R002
0003AbcdData for R003
...

However, the VSAM DATASET (which was created with key offset 0, and key size 8), actually ends up with:

R001    Data for R001
R002    Data for R002
R003    Data for R003
...

Despite the fact that we are writing from recArea, which includes the recID and recKeyC fields (numeric and character respectively), only the non-key portion of the WRITE “data payload” ends up hitting the disk.

If that is the case, where do the spaces in the key-portion of the records come from? Again, the C programmer in me is interfering with my understanding. I look at:

dcl keyList(10) char(8);
keyList(1) = 'R001';

and think that keyList(1) = “R001\x00\x00\x00\x00”, but it must actually be space filled in PL/I! This seems to be confirmed emperically, based on the expected results for the test, but I can also see it in the debugger after manually relocating the 32-bit mainframe address:

(gdb) p keyLen
$1 = 8
(gdb) p /x aKey + 0x7ffbc4000000
$2 = 0x7ffbc5005740
(gdb) set target-charset EBCDIC-US
(gdb) p (char *)$2
$3 = 0x7ffbc5005740 "R001    R002    R003    R004    R005    R006    R007    R008    R009    R010    "

The final form of the records in the VSAM DATASET (mainframe for a file), is now fully understood. Note that the data disagrees with the PICTURE specification for the recID field in the recData variable declaration, but that’s okay, at least for this part of the program, since there is never any store to that field that is non-numeric. Would anything even have to have been written to recID or recKeyC … I suspect not? Once we have R00N in that part of the record what happens if we read it into recData with the numeric only PICTURE specification? Does that raise a PL/1 condition?

ps. Notice how the payload for the keyList array entries is nicely packed into memory. This is done in a very non-C like fashion with no requirement for an array of pointers and the corresponding cache hit loss those pointers create when accessing a big multilevel C array.

VSAM creation and population with JCL and IDCAMS

March 7, 2017 Mainframe , , , , , , , ,

I learned a few JCL DATASET related things yesterday that seemed notable, at least for a JCL newbie.

Delete a DATASET, and ignore any error.

Each time I’ve wanted a DATASET cleanup step in JCL I’ve been using a separate script, and running that first.  A better way of doing this is to include a IDCAMS job step in the script, and have that do the deletion

//CLEANUP EXEC PGM=IDCAMS
//SYSIN DD *
  DELETE PJOOT.XXXXX005
  SET MAXCC = 0
/*
//SYSPRINT DD SYSOUT=*
//SYSOUT   DD SYSOUT=*
//*SYSTERM  DD SYSOUT=*

This deletes the file PJOOT.XXXXX005, which in this case was a VSAM file. In case that file (a DATASETs in mainframe-eze) did not exist, the error code for that DELETE is ignored by setting MAXCC=0. If you have multiple things that you want to do with IDCAMS, you can do things like DELETE and then ALLOCATE immediately, such as

//REALLOC EXEC PGM=IDCAMS
//SYSIN DD *
  DELETE PJOOT.XXXXX005
  SET MAXCC = 0
  DEFINE CLUSTER (NAME(PJOOT.XXXXX005) -
               CYLINDERS(1) VOLUMES(LZ0000) -
               INDEXED -
               KEYS(4 0) -
               RECORDSIZE(240 240) -
               ) -
         DATA (NAME(PJOOT.XXXXX005.DATA)) -
         INDEX (NAME(PJOOT.XXXXX005.INDEX))
/*
//SYSPRINT DD SYSOUT=*
//SYSOUT   DD SYSOUT=*
//*SYSTERM  DD SYSOUT=*

This does the DELETE, ignores any error, and then proceeds to do the new ALLOCATE for the VSAM file. I haven’t seen any way described of ALLOCATING a VSAM file other than using IDCAMS, except in 3270 screens. I think I’ve seen that LzLabs has 3270 capabilities for this sort of stuff, but I’m not inclined to try to figure out how to use it. I’d rather use our much more intuitive GUI or do it in script with JCL like this.

Copy a DATASET.

Here is some JCL to copy an (INLINE) dataset into the VSAM file created above

//COPY2VS EXEC PGM=IDCAMS
//TARGET DD DSN=PJOOT.XXXXX005,DISP=(OLD,KEEP,KEEP)
//INLINEDD DD DATA,DCB=(BLKSIZE=240,LRECL=240,RECFM=F)
a
brown
fox
quick
/*
//SYSIN DD *
REPRO -
  INFILE(INLINEDD) -
  OUTFILE(TARGET)
/*
//SYSPRINT DD SYSOUT=*
//SYSOUT   DD SYSOUT=*
//SYSTERM  DD SYSOUT=*

There are two quirks that are noteworthy here.

  1. The VSAM file requires the input be sorted, which is why the words from ‘a quick brown fox’ are in the explicitly sorted order above.
  2. The VSAM file was created with RECORDSIZE 240, so the input file had to be forced to LRECL=240 to match.

Omission of either sort or the LRECL matching causes the VSAM load to fail.

This was the first time that I’d seen this specific INLINE DD syntax, with explicit parameters.  The way I’d seen it before was how SYSIN was specified above with ‘NAME DD *’, ending with C “comment start” /* sequence.  It turns out the default end of file delimiter can also be specified, for example, this also works:

//INLINEDD DD DATA,DLM=@@,DCB=(BLKSIZE=240,LRECL=240,RECFM=F)
a
brown
fox
quick
@@

Cat a file to spool

Because IDCAMS can copy files, this can also be used to cat a file to SPOOL if desired.  Here’s an example:

//CATVS JOB
//CATVS EXEC PGM=IDCAMS
//TARGET DD DSN=PJOOT.XXXXX005,DISP=(OLD,KEEP,KEEP)
//SYSIN DD *
REPRO -
  INFILE(TARGET) -
  OUTFILE(SYSOUT)
/*
//SYSPRINT DD SYSOUT=*
//SYSOUT   DD SYSOUT=*
//SYSTERM  DD SYSOUT=*

If I include a step like this, I’m able to see the file contents in our nice GUI spool browser along with the JCL script and all the other output.