Trying to understand Suitcase

[CyrilleFranchet]

Hi, I'm trying to use Suitcase to define the Apple iAP protocol received through a stream.

I'm not sure my approach to handle the payload length is smart enough. The size could be on 1 byte or 3bytes depending if the packet is a small one or a large one. I used ConditionalField but would have preferred to record the length in only one field.

I also don't understand why I can't use Structure this way without fixing the length. I'm not able to decode LingoGeneralIdentifyDeviceLingoes as soon as I use DispatchTarget() in LingoGeneral.

class LingoGeneralIdentifyDeviceLingoes(Structure):
    id = UBInt8Sequence(12)


class LingoGeneral(Structure):
    command_id = DispatchField(UBInt8())
    command_data = DispatchTarget(dispatch_field=command_id, dispatch_mapping={
      0x13: LingoGeneralIdentifyDeviceLingoes,
    })


class LingoSimpleRemote(Structure):
    command_id = DispatchField(UBInt8())
    command_data = Payload()


class LingoDisplayRemote(Structure):
    command_id = DispatchField(UBInt8())
    command_data = Payload()


class LingoExtendedInterface(Structure):
    command_id = DispatchField(UBInt16())
    command_data = Payload()


class LingoPacket(Structure):
    header = Magic(b'\xFF\x55')
    payload_length = LengthField(UBInt8(), get_length=lambda l: l.getval() - 1, set_length=lambda f, v: f.setval(v + 1))
    large_payload_length = ConditionalField(LengthField(UBInt16(), get_length=lambda l: l.getval() - 1,
                                                  set_length=lambda f, v: f.setval(v + 1)), lambda m: m.payload_length == 0)
    lingo_id = DispatchField(UBInt8())
    payload = ConditionalField(DispatchTarget(length_provider=payload_length, dispatch_field=lingo_id, dispatch_mapping={
        0x00: LingoGeneral,
        0x02: LingoSimpleRemote,
        0x03: LingoDisplayRemote,
        0x04: LingoExtendedInterface
    }), lambda m: m.payload_length != 0)
    large_payload = ConditionalField(
        DispatchTarget(length_provider=large_payload_length, dispatch_field=lingo_id, dispatch_mapping={
            0x00: LingoGeneral,
            0x02: LingoSimpleRemote,
            0x03: LingoDisplayRemote,
            0x04: LingoExtendedInterface
    }), lambda m: m.payload_length == 0)
    checksum = CRCField(UBInt8(), algo=ipod_checksum, start=2, end=-1)

[mikewadsten]

The size could be on 1 byte or 3bytes depending if the packet is a small one or a large one. I used ConditionalField but would have preferred to record the length in only one field.

Since I am not familiar with the iAP protocol (and can't seem to find any documentation for it online), I'll have to make an educated guess based on your post.

If the payload length (e.g. 1 or 3 bytes) is actually called out in the message, then you should be able to just use it as a normal LengthField. Things can get tricky if the length field is in the top-level structure but you need it in a substructure, but I believe that can be solved using DependentField.

Otherwise, if the length is determined by some other type code (e.g. if lingo_id is X then it's 1, if lingo_id is Y then it's 3), dispatching to substructures that use UBInt8Sequence to capture the payload may also be suitable.

Another approach that we have used on occasion, when Suitcase's protocol parser wasn't quite up to the task, was e.g. to split the stream into messages manually, use a high-level structure to do basic parsing, and then explicitly unpack parts of the body as needed. For example, if a 20-byte chunk of the body is hard to express in Suitcase, just capturing it as a Payload or a UBInt8Sequence and handling it later can be more expedient than trying to have Suitcase do all the heavy lifting.

I would also suggest you review the unit tests, because we have some examples of complex structures that you could use as a reference.

[CyrilleFranchet]

Thanks for your reply Mike. The subtlety I don't get is that my length field has two names. How can I factor them in suitcase? It would be easier to get the length with DependentField.

[mikewadsten]

I'm not sure what you mean when you say your length field has two names. Can you post a few example message bodies (annotated to explain what fields they are)?

Or I guess, looking at your original post... do you mean that the size of the length field can change, depending on the value? As in, it can either be a single 8-bit value, or if the value is 0, then the actual length is a 16-bit value that follows? If that is the case, I might have an example of a variable-size length field that I can scrounge up.

[CyrilleFranchet]

I'm interested by your example. Because that's exactly the point. Small packets have a 1 byte length field. Large packets have a length field equals zero. In this case the length is in the two next bytes.

[mikewadsten]

@CyrilleFranchet I did a bit of digging in the most recent Suitcase-based code I had worked on, and at this time I don't believe that Suitcase can be used to elegantly express the packets you have described, at least not much further than in the example code you posted.

If the example code that you posted works, and you're mostly frustrated by the fact that accessing the length value requires you to "know" whether to look at .payload_length or .large_payload_length, you could work around that by adding something like:

    @property
    def length(self):
        if self.payload is not None:
            return self.payload_length
        else:
            return self.large_payload_length

That way, you can just use .length to access the payload length value. (You might consider using names like length_u8 and length_u16 to make the intent clearer.)

Does this help you?

[CyrilleFranchet]

Hi @mikewadsten. Finally I was able to dissect packets correctly by changing my method.
Now I'm trying to create packets but unfortunately I'm not able to get length field working.
Suitcase gives me error when I try to set the field myself but otherwise the field value is not calculated.

Maybe my data structure is too complicated for suitcase?

[CyrilleFranchet]

Hi @mikewadsten do you have any idea how can I force suitcase to calculate my field value?