#1
  1. No Profile Picture
    Registered User
    Devshed Newbie (0 - 499 posts)

    Join Date
    Mar 2004
    Posts
    6
    Rep Power
    0

    Reading binary using a struct - behavour not as expected?


    Hi,
    I am writing code to deal with PCAP files. I have a PCAP dump and I am looking at the timestamps in the PCAP packet headers to see if they are in the correct order in the file. To do this I have a class called PCAPPacketHdr as follows
    Code:
    import struct
    
    class PCAPPacketHdr:
        FormatString = "LLLL"
        TSSec = None
        TSUSec = None
        InclLen = None
        OrigLen = None
    
        def Pack(self):
            return struct.pack( self.FormatString, self.TSSec, self.TSUSec, self.InclLen, self.OrigLen )
    
        def Unpack(self, buffer):
            self.TSSec, self.TSUSec, self.InclLen, self.OrigLen = struct.unpack( self.FormatString, buffer )
    
        def Size(self):
            return struct.calcsize (self.FormatString)
    I then have code which opens up the file (skipping the PCAP file magic number and PCAP file header), and reads in each packet header as follows:

    Code:
    while not eof:
         #read in PCAPPacketHdr
         buf = curFile.read(packetHdr.Size())
         if len(buf) == packetHdr.Size():
             packetHdr.Unpack(buf)
             if lastPacketHdr != None:
                 if lastPacketHdr.TSSec > packetHdr.TSSec:
                     outputFile.write("ERROR: Packet TSSec earlier than last one: \n")
                     outputFile.write("       Last Packet "+repr(lastPacketHdr.TSSec)+"."+repr(lastPacketHdr.TSUSec)+"\n")
                 elif lastPacketHdr.TSSec == packetHdr.TSSec:
                     if lastPacketHdr.TSUSec > packetHdr.TSUSec:
                         outputFile.write("ERROR: Packet TSUSec earlier than last one\n")
                         outputFile.write("       Last Packet "+repr(lastPacketHdr.TSSec)+"."+repr(lastPacketHdr.TSUSec)+"\n")
             outputFile.write("       Packet "+repr(packetHdr.TSSec)+"."+repr(packetHdr.TSUSec)+"\n")
             lastPacketHdr = copy.deepcopy(packetHdr)
             #skip packet payload
             packetPayload = curFile.read(packetHdr.InclLen)
         else:
             eof = True
    This code appears to work fine for extracting the timestamps from the file, the repr( ) calls on the timestamps allow me to write them to the output file correctly, it's just the comparison operators don't appear to be working as I would expect. It appears than when the TSUSec timestamp is the same as the previous one in the data I input, it reports "ERROR: Packet TSUSec earlier than last one".

    This makes me think that the comparison operators aren't acting on the data as longs as I expected.

    Can anyone shed some light on what I'm doing wrong, I'm still very new to Python.

    Thanks in advance,

    Rich
  2. #2
  3. Mini me.
    Devshed Novice (500 - 999 posts)

    Join Date
    Nov 2003
    Location
    Cambridge, UK
    Posts
    783
    Rep Power
    13
    Hi
    The only thing that looks unusual to me is
    Code:
    lastPacketHdr = copy.deepcopy(packetHdr)
    I haven't used copy.deepcopy on classes myself but the notes suggest that __getinitargs__(), __getstate__() and __setstate__() methods should be implemented to ensure the instance data is copied properly.

    why not use this instead:
    Code:
    lastPacketHdr = packetHdr
    packetHdr = PCAPPacketHdr()
    Grim
  4. #3
  5. No Profile Picture
    Contributing User
    Devshed Intermediate (1500 - 1999 posts)

    Join Date
    Feb 2004
    Location
    London, England
    Posts
    1,585
    Rep Power
    1373
    I am not familiar with the PCAP data structure, but it is possible that the byte order is different than the order that struct.unpack is using.

    By default the struct module uses the byte order native to the system you are running on, so could be little endian or big endian. You can force it to use one or the other by starting the format string with '<' or '>'.

    There may also be padding bytes in the PCAP format that are not represented in the struct format, or vice-versa. This would cause your code to get out of sync with the data. However I think this is less likely since you are always reading out longs, unless the 'packetPayload' bytes that you skip over do not include padding bytes.

    Have you tried printing out the values you are reading to see if they look reasonable?

    Dave - The Developers' Coach

IMN logo majestic logo threadwatch logo seochat tools logo