DevOps

How to Use Python Struct Functions

How to Use Python Struct Functions

Introduction

Python structs are a practical tool for managing and storing raw byte data. The struct module enables the creation of compact binary data types used in the C programming language.

Structs efficiently store and work with fixed-size data in memory. This tool allows transferring data between systems and languages which use binary data representations.

This article shows how to use the Python struct module and the available functions.

How to Use Python Struct Functions

Prerequisites

Python Struct Module Overview

The Python struct module has two functionalities:

  • Data exchange with external sources, such as network transfers.
  • Data transfer between Python and C applications.

Structs require constructing a format string for packing and unpacking data. The character combinations describe the data type being packed and unpacked.

The table below describes some commonly used characters object types in Python and C, with their corresponding sizes in bytes.

Character Python Type C Type Size
c bytes length 1 char 1
? bool _Bool 1
i integer int 4
l integer long 4
q integer long long 8
f float float 4
d float double 8
s bytes char[] Corresponds to the number of characters.

For example, the format string for three numbers (such as 100, 200, 300) is iii or 3i, while the size is 12 bytes (4 for each integer).

Apart from the string format, some struct functions require adding a buffer. The buffer is commonly a bytes() or bytearray() object.

Python Struct Functions

The Python struct module offers several functions to pack and unpack structs and an error catcher for exceptions. The examples below demonstrate how to use each functionality from the struct module through code.

All examples require importing the module:

import struct

The struct library comes with Python by default.

Note: In case you haven’t installed Python yet, use one of our OS-based guides:

struct.pack()

The struct.pack() function packs values according to the string format. The syntax is:

struct.pack("format", value_1, value_2, etc)

The format string must exactly match the value format. For example:

import struct

packed_struct = struct.pack("3i", 1, 2, 3)
print(packed_struct)
struct.error python output

The format string (3i) corresponds to the provided values (three integers). As a result, the code prints the packed struct, where each value is 4 bytes.

Note: If working with textual data, use byte literals instead of string literals. For example:

b"abc"

struct.unpack()

The struct.unpack() function unpacks a packed struct to the original format. The syntax is:

struct.unpack("format", packed_struct)

The format is identical to the one used to pack a struct. For example:

import struct

packed_struct = struct.pack("3i", 1, 2, 3)
print("Packed struct:")
print(packed_struct)

unpacked_struct = struct.unpack("3i", packed_struct)
print("Unpacked struct:")
print(unpacked_struct)
struct.error python output

The code packs a struct with three integers and unpacks it using the same string format. The unpacked struct is in a list.

struct.calcsize()

The struct.calcsize() function helps calculate the string format byte size. To see the size, pass the string format as an input value. For example:

import struct

print(struct.calcsize("3i"))
struct.error python output

The code shows the size for 3i as 12. Each integer is four bytes, so 12 is the total size for three integers.

struct.pack_into()

The struct.pack_into() function packs a struct into a buffer. The syntax is:

struct.pack_into("format", buffer, offset, value_1, value_2, etc)

The buffer is either a bytes() object for a single value or a bytesarray() for multiple values. The minimum buffer size is the total bytes of the values. If the buffer size exceeds the data size, the offset value allows shifting the starting data position. All empty bytes are zero-filled.

For example:

import struct

buffer = bytearray(13)
print("Empty buffer:")
print(buffer)

struct.pack_into("3i", buffer, 1, 2, 3, 4)
print("Populated buffer")
print(buffer)

The code creates an empty bytesarray() object with 13 bytes. Initially, the array contains zeros to serve as placeholders.

Note: Use the struct.calcsize() function to calculate the buffer size for complex string formats. If using an offset, the total byte size is the offset plus the struct.calsize() result.

The struct_packinto() function packs data into the buffer with offset one. The object fits three integer values (four bytes each), with the first value being zero to account for the shift.

struct.error python output

The output prints the empty buffer, followed by the populated buffer.

struct.unpack_from()

The struct.unpack_from() function unpacks a populated buffer. The syntax is:

stuct.unpack_from("format", buffer, offset)

Omit the offset parameter if the original offset is zero.

The following code shows how to pack a struct into a buffer and unpack the values:

import struct

buffer = bytearray(13)
print("Empty buffer:")
print(buffer)

struct.pack_into("3i", buffer, 1, 2, 3, 4)
print("Populated buffer")
print(buffer)

unpacked_buffer = struct.unpack_from("3i", buffer, 1)
print("Unpacked buffer")
print(unpacked_buffer)
struct.error python output

The result unpacks the original data from the struct in a list.

struct.iter_unpack()

The struct.iter_unpack() function unpacks a struct into an iterative format instead of a ready list. The syntax is:

struct.iter_unpack("format", buffer, offset)

Use a loop to iterate through the values in the unpacked struct. For example:

import struct

buffer = bytearray(12)
print("Empty buffer:")
print(buffer)

struct.pack_into("3i", buffer, 0, 2, 3, 4)
print("Populated buffer")
print(buffer)

iterative_unpack = struct.iter_unpack("3i", buffer)

for value in iterative_unpack:
    print(value)
struct.error python output

The for loop goes through all the values in the iterative struct and prints them to the output.

Exception struct.error

The struct library contains a function that stores the error message. Use the struct.error exception in a try except statement to parse the message.

For example:

import struct

buffer = bytearray(2)

try:
    pack_buffer = struct.pack_into("3i", buffer, 0, 2, 3, 4)
except struct.error as e:
    print("I've caught an error! Here it is: ", e)
struct.error python output

The code creates a two-byte buffer, which is not enough to store three integers. As a result, the exception handles the error and prints the message as output.

Conclusion

After reading this guide and working through the examples, you know how to pack and unpack structs. Structs help pack and unpack data in a consistent format.

Đăng ký liền tay Nhận Ngay Bài Mới

Subscribe ngay

Cám ơn bạn đã đăng ký !

Lỗi đăng ký !

Add Comment

Click here to post a comment

Đăng ký liền tay
Nhận Ngay Bài Mới

Subscribe ngay

Cám ơn bạn đã đăng ký !

Lỗi đăng ký !