SolarOrbiter PAS

CCSDS library

We created at IRAP a python library to parse CCSDS TM files.

This tools is able to parse a sequence of CCSDS TM/TC packets, from binary or hexadecimal file.

Installation notes

This library needs a python 3.6.x or greater.

Spice kernels

It uses also another library, to handle Spice Kernels files to provide correct timetags from CCSDS coarse_time and fine_time fields.

See software/python/spice_manager.py

This tool depends on an external library, spiceypy that can be installed using:

pip instal spiceypy

MIB database

The CCSDS library needs also an access to Solar orbiter MIB database, to extract some useful informations.

This MIB database can be downloaded from MSSL or SOC website.

See software/python/mib.py

The various MIB database versions have to be installed in a directory, corresponding to MIB_DIR shell environmment variable.

export MIB_DIR=/DATA/SOLAR/MIB

And we need also to create a Linux logical link to the latest MIB available version

$ cd $MIB_DIR
$ ln -s MIB__20210330M08921PFMS001_SOL latest

CCSDS packet attributes

For each packet TM/TC packet, we can have access to the following attributes

packet_type	Packet type (O: TC, 1: TM)
pid	Identifier (95..99) for SWA instruments
categ	Category of packets
apid	APID = 16 PID + categ
seq	Sequence counter (for each apid)
service_type	PUS service type
sub_service	PUS sub type
coarse	coarse time (integer seconds)
fine	fine time (integer x 1/512s)
dt	timetag (python datetime) computed with SpiceKernels correction
size	Size in bytes of paylod
data	payload byte array
length	CCSDS length field
raw	byte array (including CCSDS headers)
sid	Packet identifier (214 for PAS calibration)
spid	Packet identifier in MID database
descr	Packet description extracted from MIB
tc_name	ZIAxxxx
tc_descr	TC description from MIB

CCSDS file reader

This function allows to parse a binary or hexadecimal CCSDS file, as a sequence of TM/TC packets.

A first example, to parse the whole file, selecting TC_ACK TM packet (service_type = 1) and display default packet attributes

from CCSDS import CCSDS_reader

filename = "20210424_000000_20210425_000000.swa-batch-tm.bin"

for packet in CCSDS_reader (filename):

        if packet.service_type == 1:
                print (packet)

Produces the following output

APID( 99, 1) TM(  1,  1) # 1359 Size =     4 2021-04-24T05:27:30.701   SWA_TM_CMD_ACP
APID( 99, 1) TM(  1,  1) # 1359 Size =     4 2021-04-24T05:27:30.701   SWA_TM_CMD_ACP
APID( 99, 1) TM(  1,  7) # 1360 Size =     4 2021-04-24T05:27:33.951   SWA_TM_CMD_EXE
APID( 99, 1) TM(  1,  7) # 1360 Size =     4 2021-04-24T05:27:33.951   SWA_TM_CMD_EXE
APID( 99, 1) TM(  1,  1) # 1361 Size =     4 2021-04-24T05:27:34.094   SWA_TM_CMD_ACP
APID( 99, 1) TM(  1,  1) # 1361 Size =     4 2021-04-24T05:27:34.094   SWA_TM_CMD_ACP
APID( 99, 1) TM(  1,  7) # 1362 Size =     4 2021-04-24T05:27:34.244   SWA_TM_CMD_EXE
APID( 99, 1) TM(  1,  7) # 1362 Size =     4 2021-04-24T05:27:34.244   SWA_TM_CMD_EXE
APID( 99, 1) TM(  1,  1) # 1363 Size =     4 2021-04-24T05:27:34.244   SWA_TM_CMD_ACP
APID( 99, 1) TM(  1,  1) # 1363 Size =     4 2021-04-24T05:27:34.244   SWA_TM_CMD_ACP
APID( 99, 1) TM(  1,  7) # 1364 Size =     4 2021-04-24T05:27:34.368   SWA_TM_CMD_EXE
APID( 99, 1) TM(  1,  7) # 1364 Size =     4 2021-04-24T05:27:34.368   SWA_TM_CMD_EXE
APID( 99, 1) TM(  1,  1) # 1365 Size =     4 2021-04-24T12:00:00.777   SWA_TM_CMD_ACP
APID( 99, 1) TM(  1,  7) # 1366 Size =     4 2021-04-24T12:00:00.777   SWA_TM_CMD_EXE
APID( 99, 1) TM(  1,  1) # 1367 Size =     4 2021-04-24T12:00:10.786   SWA_TM_CMD_ACP
APID( 99, 1) TM(  1,  7) # 1368 Size =     4 2021-04-24T12:07:25.028   SWA_TM_CMD_EXE

We can modify the default behaviour, replacing the default output with:

        if packet.service_type == 1:
                print (packet.raw.hex())

Will display the whole TM packet, packet.raw, but converted to hexadecimal with the hex() function.

0e31c54f000d1001016e28166883c0a11dfccfcf
0e31c54f000d1001016e28166883c0a11dfccfcf
0e31c550000d1001076e2816688700991dfccfcf
0e31c550000d1001076e2816688700991dfccfcf
0e31c551000d100101fe2816688725611dfcc000
0e31c551000d100101fe2816688725611dfcc000
0e31c552000d100107fe281668874b981dfcc000
0e31c552000d100107fe281668874b981dfcc000
0e31c553000d100101fe281668874ba41dfcc000
0e31c553000d100101fe281668874ba41dfcc000
0e31c554000d100107fe281668876b641dfcc000
0e31c554000d100107fe281668876b641dfcc000
0e31c555000d1001016e2816c481c9e61dfccfd0
0e31c556000d1001076e2816c481ca071dfccfd0
0e31c557000d1001016e2816c48bcc4e1dfccfd1
0e31c558000d1001076e2816c63e0a2f1dfccfd1

We can also produce a binary output file, from a CCSDS hexadecimal file and select only SWA_EVENTS.

input_file = "20210424_000000_20210425_000000.swa-batch-tm.hex"

output_file = "20210424.event.bin"

with open (output_file, "wb") as output:

        for packet in CCSDS_reader (input_file):

                if packet.service_type == 5:
                        output.write (packet.raw)

TM data filtering

It is possible to filter TM/TC data using dedicated filtering functions.

Filtering SWA EVENTS packet

def     event_filter (packet):
        return packet.service_type == 5 and packet.sub_service in [1,2,3,4]

Filtering TC acknowledgements

def     ack_filter (packet):
        return packet.service_type == 1 and packet.sub_service in [7,8]

Filtering PAS calibration data

def     calib_filter (packet):
        return packet.apid == 1532 and packet.sid == 214

A more complex example:

the following function start to convert a string parameter to a internal python datetime.date
then compute start and stop variables (python datetime objects) to cover from 00:00 to 24:00
then creates a filtering function that embed these start/stop time interval to filter TM packets
finally, return the filtering function

def     make_daily_filter (str):
        """
        Return a function that filters daily TM packets
        """
        day = str_to_date (str)
        start = datetime.datetime (day.year, day.month, day.day, 0, 0, 0)
        stop  = start + datetime.timedelta (days = 1)

        def     filter (packet):
                return start <= packet.dt < stop

        return filter

Then we can use these filtering functions to select the corresponding packets.

from CCSDS import CCSDS_reader

filename = "20210424_000000_20210425_000000.swa-batch-tm.bin"

# choose one from
my_filter = event_filter
my_filter = ack_filer
my_filter = make_daily_filter ("2021-04-24")

for packet in filter (my_filter, CCSDS_reader (filename)):

        print (packet)

Merge/select data from several input files

It should be possible to merge several input files, applying some filter, to produce an unique output file.

One example of use is to merge various CCSDS TM file, corresponding to daily files downloaded from EDDS server with storage_time parameter.

We can run every day a new request to EDDS to download all SWA data with storage_time in range 00:00/24:00.

That will create daily files with all SWA packets dumped during this period.

Then, we can merge all these files and filter CCSDS packet using their timetag (corresponding to CCSDS packet generation_time), in a given daily range.

So we can reconstruct a daily generation_time TM file.

def     merge_TM (input_files):
        """
        Produce a sequence of TM packet from a list of CCSDS input files
        """
        for filename in input_files:

                for packet in CCDS_reader(filename):

                        yield packet

def     produce_TM_file (str_date, input_files):
        """
        Produce daily TM file from a list of input files
        """
        my_filter = make_daily_filter (str_date)

        output_file = "%s.generation_time.bin" % str_date

        with open (output_file, "wb") as output:

                for packet in filter (my_filter, merge_TM (input_files)):

                        output.write (packet.raw)

        print ("Generation", output_file)


if __name__ == "__main__":

        from sys import argv

        day = argv [1]

        input_files = argv [2:]


        produce_TM_file (day, input_files)

Example of use:

$ python produce_TM.py 2021-04-08 20210408.storage_time.bin 20210409.storage_time.bin 20210410.storage_time.bin 20210411.storage_time.bin ...

Solar Orbiter PAS

CCSDS tools