Writing a File Receiver

File receivers reassemble files that a satellite sends in chunks, one chunk per frame. gr-satellites provides a generic FileReceiver class that handles bookkeeping (which file a chunk belongs to, where it goes, when a file is complete), so a receiver for a new protocol only describes the chunk format. This page documents that class's contract, the ImageReceiver variant for live image display, a worked example and how to register it.

How It Fits

A SatYAML files: <name> or image: <name> entry creates a file receiver data sink, which instantiates satellites.filereceiver.<name> as:

receiver_class(path, verbose, **kwargs)

and calls push_chunk(frame_bytes) for every frame it receives. For image: entries, display=True is passed; the image receiver then shows the image with feh.

The FileReceiver Contract

Subclass satellites.filereceiver.filereceiver.FileReceiver and override the methods your protocol needs. Every method that receives chunk receives the value returned by parse_chunk().

Method	Default	Override to
`parse_chunk(chunk)`	returns `chunk` unchanged	Validate and parse the raw bytes. Return `None` to drop the frame (for example a telemetry frame that is not a file chunk).
`file_id(chunk)`	`chunk.file_id` if present, else `None`	Identify which file the chunk belongs to.
`chunk_sequence(chunk)`	`chunk.sequence` if present, else `None`	Return the chunk index, counting from 0.
`chunk_size()`	`None`	Return the fixed chunk size if all chunks (except possibly the last) are equal.
`chunk_offset(chunk)`	`chunk.offset`, else `chunk_size() * chunk_sequence()`, else `None`	Return the byte offset of the chunk in the file.
`chunk_data(chunk)`	`chunk.data`	Return the payload bytes.
`file_size(chunk)`	`chunk.filesize` if present	Return the total file size in bytes, if known.
`file_chunks(chunk)`	`chunk.chunks` if present	Return the total number of chunks, if known.
`is_last_chunk(chunk)`	`None`	Return `True` for the final chunk if the protocol marks it.
`filename(fid)`	`str(fid)`	Name of the file for a file ID.
`on_completion(f)`	does nothing	Act on a finished file (`f.path` is a `pathlib.Path`).

The defaults read attributes, so a construct Struct with fields named file_id, sequence, offset, data, filesize or chunks works with no overrides other than parse_chunk().

What push_chunk() does

Parses the chunk; drops it if parse_chunk() returns None.
Chooses the file: file_id(), else the file currently being received, else a new counter value.
If the protocol has no file IDs and the sequence number goes down, the current file is considered finished and a new one is started.
Determines the write offset from chunk_offset(). If there is no offset, it needs consecutive sequence numbers: a gap marks the file as broken and later chunks of it are ignored. If neither offset nor sequence can be computed, it raises Exception('FileReceiver unable to compute chunk_offset() nor chunk_sequence()').
Rejects chunks that would write past a known file_size().
Writes the data and flushes the file.
Calls on_completion() when the size, the chunk count or is_last_chunk() says the file is complete.

Files are opened in the output directory with mode r+b if they already exist (they are not truncated) or wb otherwise. File names with path components are replaced by a random UUID.

ImageReceiver

satellites.filereceiver.imagereceiver.ImageReceiver extends FileReceiver with:

__init__(path, verbose=False, display=False, fullscreen=True),
a default filename(fid) of f'{fid}.jpg',
live display: once 640 bytes of the current file have been written, it starts feh (fullscreen unless fullscreen=False, reloading every second) on the file.

Use ImageReceiver as the base class for anything referenced by an image: entry.

Example

The bundled Lucky-7 receiver shows a typical fixed-chunk protocol. Each chunk has one leading byte, then a 2-byte address, a 2-byte sequence number and a 2-byte chunk count (all big-endian), then 28 bytes of image data:

import struct

from .imagereceiver import ImageReceiver


class ImageReceiverLucky7(ImageReceiver):
    def chunk_sequence(self, chunk):
        return struct.unpack('>H', chunk[3:5])[0]

    def chunk_size(self):
        return 28

    def chunk_data(self, chunk):
        return chunk[7:]

    def file_size(self, chunk):
        return self.chunk_size() * struct.unpack('>H', chunk[5:7])[0]

    def parse_chunk(self, chunk):
        address = struct.unpack('>H', chunk[1:3])[0]
        if (address < 0xC000
                or address >= (
                    0xC000 + self.file_size(chunk)/self.chunk_size())):
            return None
        return chunk


lucky7 = ImageReceiverLucky7

There is no file ID, so files are separated by the sequence number dropping back; the offset is 28 × sequence; and parse_chunk() drops frames whose address is outside the image range (telemetry frames). The module ends by binding the lower-case name used in SatYAML files.

Registering

Save the module in python/filereceiver/.
Import the name in python/filereceiver/__init__.py, for example from .my_sat import my_sat.
Add the file to python/filereceiver/CMakeLists.txt.
Reference it from a SatYAML files: or image: entry.

Testing

Feed chunks directly:

from satellites.filereceiver import lucky7

rx = lucky7('/tmp/images', verbose=True)
for chunk in chunks:          # bytes objects, e.g. from a KISS file
    rx.push_chunk(chunk)

With verbose=True you see new file, received sequence, File reception is broken. and file ... complete messages.