Reading binary using a struct - behavour not as expected?

Dev Shed Forums Sponsor:

Get inside! Sample the range of functionality easily built with JMSL Library for Time Series Data Analysis, Heat Maps, Portfolio Optimization, Monte Carlo Simulation, Stock Price Charting and more. Download Now!

April 5th, 2004, 04:19 AM

RichardD

Registered User

Join Date: Mar 2004

Posts: 6

Time spent in forums: 3 h 49 m 27 sec
Reputation Power: 0

Reading binary using a struct - behavour not as expected?

Hi,
I am writing code to deal with PCAP files. I have a PCAP dump and I am looking at the timestamps in the PCAP packet headers to see if they are in the correct order in the file. To do this I have a class called PCAPPacketHdr as follows

Code:

import struct

class PCAPPacketHdr:
    FormatString = "LLLL"
    TSSec = None
    TSUSec = None
    InclLen = None
    OrigLen = None

    def Pack(self):
        return struct.pack( self.FormatString, self.TSSec, self.TSUSec, self.InclLen, self.OrigLen )

    def Unpack(self, buffer):
        self.TSSec, self.TSUSec, self.InclLen, self.OrigLen = struct.unpack( self.FormatString, buffer )

    def Size(self):
        return struct.calcsize (self.FormatString)

I then have code which opens up the file (skipping the PCAP file magic number and PCAP file header), and reads in each packet header as follows:

Code:

while not eof:
     #read in PCAPPacketHdr
     buf = curFile.read(packetHdr.Size())
     if len(buf) == packetHdr.Size():
         packetHdr.Unpack(buf)
         if lastPacketHdr != None:
             if lastPacketHdr.TSSec > packetHdr.TSSec:
                 outputFile.write("ERROR: Packet TSSec earlier than last one: \n")
                 outputFile.write("       Last Packet "+repr(lastPacketHdr.TSSec)+"."+repr(lastPacketHdr.TSUSec)+"\n")
             elif lastPacketHdr.TSSec == packetHdr.TSSec:
                 if lastPacketHdr.TSUSec > packetHdr.TSUSec:
                     outputFile.write("ERROR: Packet TSUSec earlier than last one\n")
                     outputFile.write("       Last Packet "+repr(lastPacketHdr.TSSec)+"."+repr(lastPacketHdr.TSUSec)+"\n")
         outputFile.write("       Packet "+repr(packetHdr.TSSec)+"."+repr(packetHdr.TSUSec)+"\n")
         lastPacketHdr = copy.deepcopy(packetHdr)
         #skip packet payload
         packetPayload = curFile.read(packetHdr.InclLen)
     else:
         eof = True

This code appears to work fine for extracting the timestamps from the file, the repr( ) calls on the timestamps allow me to write them to the output file correctly, it's just the comparison operators don't appear to be working as I would expect. It appears than when the TSUSec timestamp is the same as the previous one in the data I input, it reports "ERROR: Packet TSUSec earlier than last one".

This makes me think that the comparison operators aren't acting on the data as longs as I expected.

Can anyone shed some light on what I'm doing wrong, I'm still very new to Python.

Thanks in advance,

Rich

April 6th, 2004, 04:19 AM

Grim Archon

Mini me.

Join Date: Nov 2003

Location: Cambridge, UK

Posts: 783

Folding Points: 1488 Folding Title: Novice Folder

Time spent in forums: 3 Days 2 h 15 m 57 sec
Reputation Power: 7

Hi
The only thing that looks unusual to me is

Code:

lastPacketHdr = copy.deepcopy(packetHdr)

I haven't used copy.deepcopy on classes myself but the notes suggest that __getinitargs__(), __getstate__() and __setstate__() methods should be implemented to ensure the instance data is copied properly.

why not use this instead:

Code:

lastPacketHdr = packetHdr
packetHdr = PCAPPacketHdr()

Grim

__________________
*** Experimental Python Markup CGI V2 ***

April 6th, 2004, 11:41 AM

DevCoach

Contributing User

Join Date: Feb 2004

Location: London, England

Posts: 1,195

Time spent in forums: 1 Week 5 Days 13 h 34 m 6 sec
Reputation Power: 252

I am not familiar with the PCAP data structure, but it is possible that the byte order is different than the order that struct.unpack is using.

By default the struct module uses the byte order native to the system you are running on, so could be little endian or big endian. You can force it to use one or the other by starting the format string with '<' or '>'.

There may also be padding bytes in the PCAP format that are not represented in the struct format, or vice-versa. This would cause your code to get out of sync with the data. However I think this is less likely since you are always reading out longs, unless the 'packetPayload' bytes that you skip over do not include padding bytes.

Have you tried printing out the values you are reading to see if they look reasonable?

Dave - The Developers' Coach

Thread Tools	Search this Thread
Email this Page	Search this Thread: Advanced Search
Display Modes	Rate This Thread
Linear Mode Switch to Hybrid Mode Switch to Threaded Mode	Rate This Thread: