A {\it pickle} is a representation of a Modula-3 value as a stream of bytes. Writing a value as a pickle and then reading it back produces a value ``equivalent'' to the original value, in the sense that the value's non-ref fields have the same contents and that their REF and OBJECT fields are = in the same places. In other words, pickles preserve value, shape and sharing. You can write pickles for values that have cyclic references (such as doubly-linked lists), or that are arbitrary graph structures.

Pickles are used both to communicate values between two programs running at the same time and to store a value and reuse it in a program running later.

Pickles preserve ordinal values (integers and enumerations) exactly, automatically performing any byte-swapping that may be required. An exception is raised if a pickled ordinal value is too large to be represented in a program reading the pickle.

Pickles preserve floating point values as well as possible considering that the two machines may have different floating point representations.

Pickles do not preserve untraced references. Any untraced references will turn into NIL if they are pickled, regardless of their value.

Pickles preserve the names and signatures of procedures, but the code for the procedures can be different in the program writing the pickle and the program reading the pickle. Object methods are treated the same as procedures.

Sharing is preserved within pickles, but not between pickles.

Pickles are written and read by a single-pass algorithm that uses modest auxiliary tables and does not require a seekable stream.

For example, if you compile and run the following program:

      MODULE P1 EXPORTS Main;  IMPORT Pickle, IO;
        TYPE T = REF RECORD val: INTEGER END;
      BEGIN
        Pickle.Write(IO.OpenWrite("A.pickle"), NEW(T, val := 6))
      END P1.

      MODULE P2 EXPORTS Main; IMPORT Pickle, IO;
        TYPE U = REF RECORD val: INTEGER END;
        VAR v: U := Pickle.Read(IO.OpenRead("A.pickle"));
      BEGIN
        IO.Put(Fmt.Int(v.val) & "\n")
      END P2.

If you pickle a BRANDED REF INTEGER, there is no guarantee that you will be able to read it back as a BRANDED REF INTEGER, since the implicit brand can vary from program to program. Therefore, you should use explicit brands in types that will be pickled.

INTERFACE Pickle;

IMPORT Rd, Thread, Wr;

EXCEPTION Error(TEXT);

PROCEDURE Write(wr: Wr.T; ref: REFANY)
    RAISES {Error, Wr.Failure, Thread.Alerted};

Trace the data structure reachable via traced references from ref, convert it into a pickle, and write the pickle to wr, starting at the current position of wr and leaving wr positioned after the last byte written.

PROCEDURE Read(rd: Rd.T): REFANY
    RAISES {Error, Rd.EndOfFile, Rd.Failure, Thread.Alerted};

Read a pickle from rd, reconstruct a copy of the pickled data structure, and return it. The bytes to be read start at rd's current position; rd is left positioned after the last byte consumed.

\paragraph{Pickle writers and readers.} A Pickle.Writer is an object whose write method writes pickles to the writer stored in its wr data field. You can override the write method to produce special effects.

TYPE
  TypeCode = INTEGER;

Used to represent types of allocated objects. A TypeCode value is valid within a single execution of a single program. There are methods in Writer and Reader objects for writing and reading program-independent representations of a TypeCode.

TYPE
  Writer <: WriterPublic;
  WriterPublic = OBJECT
      wr: Wr.T;
    METHODS
      write(r: REFANY) RAISES
        {Error, Wr.Failure, Thread.Alerted};
      writeType(tc: TypeCode) RAISES
        {Wr.Failure, Thread.Alerted};
      writeInt(i: INTEGER) RAISES
        {Wr.Failure, Thread.Alerted};
    END;

The default method w.write(r) behaves as follows:

\begin{itemize} \item If r=NIL, write a representation of NIL on w.wr.

\item Otherwise if r has been previously written to the current pickle, write the index of r in this pickle on w.wr.

\item Otherwise, find the Special (as defined below) for the closest supertype of r's allocated type for which a special has been registered. Let sp be this special (note that there always is such a special, since the implementation registers a special for REFANY). Call writeType(sp.sc), then write a representation of TYPECODE(r), then invoke sp.write(r, w). \end{itemize}

The call w.writeType(tc) writes bytes on w.wr to represent the type tc in a program-independent fashion. These bytes are a fingerprint: fingerprints have the property that if two Modula-3 types are equivalent, then their fingerprints are equal; and with extremely hight probability, if their fingerprints are equal then they are equivalent. Note that the implementation of this method optimizes so that if a type is written to a pickle many times, occurences after the first typically occupy only one byte.

The call w.writeInt(i) writes bytes on w.wr to represent the integer i in a platform-independent fashion. In particular, the readInt method of a subsequent Pickle.Reader can read the integer correctly, even if executed on a platform with different byte order or integer size (if possible).

A Pickle.Writer may be serially re-used to pickle multiple objects.

The call Pickle.Write(wr, ref) is equivalent to (but more efficient than) the following:

      NEW(Pickle.Writer, wr := wr).write(ref)

TYPE
  Reader <: ReaderPublic;
  RefID = INTEGER;
  ReaderPublic = OBJECT
      rd: Rd.T;
    METHODS
      read(): REFANY RAISES
        {Error, Rd.EndOfFile, Rd.Failure, Thread.Alerted};
      readType(): TypeCode RAISES
        {Error, Rd.EndOfFile, Rd.Failure, Thread.Alerted};
      readInt(): INTEGER RAISES
        {Error, Rd.EndOfFile, Rd.Failure, Thread.Alerted};
      noteRef(ref: REFANY; id: RefID);
    END;

The default method r.read(r) behaves as follows:

\begin{itemize} \item If r.rd contains a representation of NIL, consume those bytes and return NIL.

\item Otherwise, if r.rd contains the index of a previously read value in this pickle, consume those bytes and return that value.

\item Otherwise compute sc = readType(), and proceed as follows. Find the registered special sp (as defined below) whose sp.sc field equals sc. Read the representation of a type code ac from r.rd. Let id be the index that the result of this method will have in the pickle. Invoke sp.read(ac, r, id), call r.noteRef(result, id), and return the result. Note that in the default case sp will be RootSpecial(), defined below. \end{itemize}

In order to implement the second case above, the Pickle.Reader maintains a table mapping indexes to references. In simple cases this table can be maintained entirely by the r.read method, which places the result of the sp.read invocation into the table. But if the value being unpickled will contain an occurence of its own reference (the second case above), the reference must be placed in the table before unpickling that occurence. To do so, the special's read method must call r.noteRef before any such occurrence. More exactly, a special's read method must call r.noteRef before any occurrence of the ref that the read method will return, giving noteRef the reference and the refID that was a parameter of the read method. If there is no such occurrence, the special need not (but may) call r.noteRef. It is a checked runtime error to call r.noteRef with a value other than the value that will be returned by the special's read method; it is OK to call r.noteRef multiple times with the same value.

The call r.readType reads bytes from r.rd that represent a type, as written by the writeType method of a Writer, and converts them into a typecode valid in the current execution of the current program. If there is no such type, or if the bytes couldn't correspond to ones written by the writeType method, raises Pickle.Error.

The call r.readInt reads bytes from r.rd that represent an integer, as written by a writer's writeInt method, and converts them into an integer, taking into consideration reperesentation differences such as byte order and integer length. It raises Pickle.Error if the integer cannot be represented on this machine.

A Pickle.Reader may be serially re-used to unpickle multiple objects.

The call Read(rd) is equivalent to (but more efficient than) the following:

      RETURN NEW(Reader, rd := rd).read();

\begin{itemize} \item the methods must leave the Rd.T or Wr.T positioned after the last byte read or written;

\item the read method must consume the number of bytes written by the write method;

\item the read method must produce a value equivalent to the one that was given to the write method. \end{itemize}

If these rules are violated, the result could be either a checked runtime error or an invalid result from reading a pickle.

There are many ways to program a special. For example, the write method could modify the value and then call the root special. Or the write method could create a related value and call writer.write or Special.write. Or it could write some data fields individually and call writer.write for selected sub-values of its value. Or it could use mixtures of these techniques.

TYPE
  SpecialPublic = OBJECT
      sc: TypeCode;
    METHODS
      write(ref: REFANY; writer: Writer)
        RAISES {Error, Wr.Failure, Thread.Alerted};
      read(reader: Reader; id: RefID): REFANY
        RAISES {Error, Rd.EndOfFile, Rd.Failure,
          Thread.Alerted};
    END;
  Special <: SpecialPublic;

The default write method calls writer.writeType(TYPECODE(ref)), then writes on writer.wr the bytes of ref's untraced data fields, then calls writer.write(x) for each traced value x directly contained in ref.

The special's read method must return an appropriate value read from rd. If the value might involve a reference to itself, the special must pass the value and id to r.noteRef before the first such occurrence of the value. In other situations the special need not (but may) call r.noteRef. See the earlier discussion of the r.read method.

The result of the read method must be a value acceptable in the context where the method is being invoked. For example, if the method is being invoked to read the contents of a field of a record, the result must be assignable to the declared type of the field. It is always correct to return a value of the same type as the value that was written when the pickle was created, but a sub-type might be acceptable too.

The default read method reads a typecode tc by calling reader.readType(), allocates a value whose type-code is tc, passes the allocated value to the reader's noteRef method, reads the bytes of its untraced data fields from reader.rd, then for each directly contained traced reference calls reader.read() and assigns the result into the appropriate field of the allocated value; finally it returns the value.

Note that the special's methods can use the writeType and readType methods of the Writer or Reader to record relevant type information. For example they could record the allocated type of the value being pickled in the same way as the default methods do, or they could record more extensive type information if available, such as the entire type ancestry of the value's type.

PROCEDURE RegisterSpecial(sp: Special);

Register sp as the special for pickling and unpickling objects having type code sp.sc.

END Pickle.

      TYPE WriterP = Pickle.Writer OBJECT
        tbl: TxtTxtTbl.T
      METHODS
        init(): WriterP := InitP
      OVERRIDES
        write := WriteP
      END;
      
      PROCEDURE WriteP(self: WriterP; r: REFANY)
        RAISES {Error, Wr.Failure, Thread.Alerted} =
        VAR txt: TEXT;
        BEGIN
          IF ISTYPE(r, TEXT) THEN
            IF self.tbl.get(r, txt) THEN 
              r := txt
            ELSE
              EVAL self.tbl.put(r, r)
            END
          END;
          Pickle.Writer.write(self, r)
        END WriteP;
      
      PROCEDURE InitP(self: WriterP): WriterP =
        BEGIN
          self.tbl := NEW(TxtTxtTbl.T).init()
          RETURN self
        END InitP;

Given the program above, you can pickle a data structure ref to the writer wr, identifying texts that have the same contents, with:

       NEW(WriterP, wr := wr).init()(ref)

Next we consider a simple example of the use of specials. Suppose that you have a type defined by

      TYPE T = REF RECORD ref: REFANY; tc: CARDINAL END

Here is one way to register specials for type T:

      TYPE TSpecial = Pickle.Special OBJECT
        OVERRIDES
          write := WriteT;
          read := ReadT
        END;

      PROCEDURE WriteT(
          self: TSpecial; 
          ref: REFANY; 
          wr: Pickle.Writer)
        RAISES {Error, Wr.Failure, Thread.Alerted} =
        VAR 
          t: T := ref; 
          fp := RTTypeFP.ToFingerPrint(t.tc); 
        BEGIN
          FOR i := 0 TO 7 DO 
            Wr.PutChar(wr.wr, VAL(fp[i], CHAR)) 
          END;
          wr.write(t.ref)
        END WriteT;

      PROCEDURE ReadT(
          ac: INTEGER; 
          rd: Pickle.Reader; 
          id: RefID): REFANY
        RAISES {Error, Rd.EndOfFile, Rd.Failure, 
                Thread.Alerted} =
        VAR fp: ARRAY [0..7] OF [0..255]; res: T; BEGIN
          FOR i := 0 TO 7 DO 
            fp[i] := ORD(Rd.GetChar(rd.rd)) 
          END;
          res := NEW(t, tc := RTTypeFP.FromFingerprint(fp));
          rd.noteRef(res, id);
          res.ref := rd.read();
          RETURN res
        END ReadT;

      ...

      Pickle.RegisterSpecial(
        NEW(TSpecial, tc := TYPECODE(T)))

As a final example, suppose that T is an object instead of a REF RECORD:

      TYPE T = OBJECT ref: REFANY; tc: CARDINAL END.

      PROCEDURE WriteT(
          self: TSpecial; 
          ref: REFANY; 
          wr: Pickle.Writer)
        RAISES {Error, Wr.Failure, Thread.Alerted} =
        VAR 
          t: T := ref; 
          fp := RTTypeFP.ToFingerPrint(t.tc); 
        BEGIN
          FOR i := 0 TO 7 DO 
            Wr.PutChar(wr.wr, VAL(fp[i], CHAR)) 
          END;
          Special.write(t, wr)
        END WriteT;

      PROCEDURE ReadT(
          ac: INTEGER; 
          rd: Pickle.Reader; 
          id: RefID): REFANY
        RAISES {Error, Rd.EndOfFile, Rd.Failure, 
                Thread.Alerted} =
        VAR 
          fp: ARRAY [0..7] OF [0..255]; 
          res: T; 
        BEGIN
          FOR i := 0 TO 7 DO 
            fp[i] := ORD(Rd.GetChar(rd.rd)) 
          END;
          res := Special.Read(ac, rd, id);
          res.tc := RTTypeFP.FromFingerprint(fp);
          RETURN res
        END ReadT;