IRAP has modified a Java StreamClient application delivered by ESA, to be able to download SolarOrbiter TM/TC in near real-time, from EDDS server.

Introduction

The new software is consists in:

  • EDDS.py

    python script, using EDDS SOAP Web Services

    • read config.properties file, to extract user authentication parameters and EDDS server address

    • connect to EDDS server with user account

    • launch a data streaming requests (TM or TC data)

    • recover the corresponding requestId

    • run Java StreamClient that gets streaming data for the corresponding request

  • StreamClientLauncher.sh

    Shell script that run the Java Stream Client to recover streaming data from EDDS server

Software requirements

  • Access to EDDS server

    Ask ESOC for an account on the EDDS server, with a valid Usename and password

    Check your network configuration rules for your computer to access EDDS server

  • python

    You need python 3.x installed on your computer, with some extra libraries :

    • zeep : to access SOAP Web services

    • isodate : ISO 8601 library (ISO time)

    • configobj : to parse config.properties

    You can use pip to install those libraries

    $ pip install zeep isodate configobj

  • java

    Java probably 8.x is necessary (to be confirmed)

Installation manual

Download and extract the /documents/EDDS/software/STREAM_CLIENT.irap.tar.gz

$ cd <install-directory>
$ tar xvzf STREAM_CLIENT.irap.tar.gz
$ cd STREAM_CLIENT.irap

You will find here:

  • config.properties :

    config file used by application to find username, password, EDDS server address

  • EDDS.py

    python EDDS library

  • StreamCLientLauncher.sh

    Shell script that run the Java StreamClient application

  • streamclient.xml

    Java beans XML config file, modified by IRAP to change ESA software behaviour

  • lib

    Original ESA Java library (.jar files)

  • src

    IRAP’s added Java source files

  • irap-streamedds.jar

    IRAP modified library (build from src/*java)

  • build

    Shell script to build the IRAP Java modified software

Configuration

You have first to update the config.properties file with your usual text editor, to fill your EDDS username and passord

config.properties
# The username that the request submitter should log on as
username = <EDDS-username>

# The password that the request submitter should log on as
password = <your-EDDS-password>

# The EDDS web server that the request submitter should submit requests to
eddsServerAddress = https://mmaasweb01.esoc.ops.esa.int:8443/edds/EddsService?wsdl

Check also the content of streamclient.xml file, to verify pathname of config.properties file

streamclient.xml
  <bean class="org.apache.camel.spring.spi.BridgePropertyPlaceholderConfigurer">
    <property name="location" value="file:///home/solar/EDDS/STREAM_CLIENT.irap/config.properties"/>
  </bean>

You can use the build script to reconstruct the IRAP Java library, but it’s not necessary if you dont’t modify the corresponding java sources.

./build

Will create/update the irap-streamedds.jar library

User Manual

From the extraction directory, just enter :

# To download TM data
$ python -m EDDS TM

# To download TC data
$ python -m EDDS TC
Caution

Actually, as SWA data is not yet available, we are using PID = 76,77 filtering for TM data test purpose

You can have a look on EDDS.py, in create_TM_stream_request() function

        StreamDataRequest = PktTmStream (
                        DataSource = "PARC",
                        Filter = "pids=76,77;sendRawData=true"
                )

We will update later the EDDS.py software, to modify PID filters, for example to filter SWA data :

Filter = "pids=95-99;sendRawData=true"

You can modify also the test_TM() or test_TC() functions:

def     test_TM ():
        """
        Create a PktTmStream request, with a given duration
        """
        duration = isodate.Duration (hours = 1)

        req = create_TM_stream_request (duration)

        job_id = send_stream_request (req)

        update_properties (job_id)

        run_stream_client()

Acknowledgements

Please acknoledge Tom.barthe@gmail.com

  • He has made available the EDDS Web services from remote python script, using SOAP protocol.

  • He has also overloaded the ESA stream client software to modify the application behaviour, in order to create a single result file per request, increasing regularly in near real-time.