EDDS Data streaming software
============================
IRAP has modified a Java StreamClient application delivered by ESA, to be able to download
SolarOrbiter TM/TC in near real-time, from EDDS server.
* link:/documents/EDDS/software/[Download STREAM_CLIENT.irap]
* link:/documents/EDDS/software/STREAM_CLIENT.irap[Browse Software directory]
== Introduction
The new software is consists in:
* *EDDS.py*
+
--
python script, using EDDS SOAP Web Services
* read config.properties file, to extract user authentication parameters and EDDS server address
* connect to EDDS server with user account
* launch a data streaming requests (TM or TC data)
* recover the corresponding requestId
* run Java StreamClient that gets streaming data for the corresponding request
--
* *StreamClientLauncher.sh*
+
--
Shell script that run the Java Stream Client to recover streaming data from EDDS server
--
== Software requirements
* Access to EDDS server
+
--
Ask ESOC for an account on the EDDS server, with a valid Usename and password
Check your network configuration rules for your computer to access EDDS server
--
* python
+
--
You need python 3.x installed on your computer, with some extra libraries :
* zeep : to access SOAP Web services
* isodate : ISO 8601 library (ISO time)
* configobj : to parse config.properties
You can use pip to install those libraries
$ pip install zeep isodate configobj
--
* java
+
--
Java probably 8.x is necessary (to be confirmed)
--
== Installation manual
Download and extract the link:/documents/EDDS/software/STREAM_CLIENT.irap.tar.gz[]
----
$ cd <install-directory>
$ tar xvzf STREAM_CLIENT.irap.tar.gz
$ cd STREAM_CLIENT.irap
----
You will find here:
* config.properties :
+
config file used by application to find username, password, EDDS server address
* EDDS.py
+
python EDDS library
* StreamCLientLauncher.sh
+
Shell script that run the Java StreamClient application
* streamclient.xml
+
Java beans XML config file, modified by IRAP to change ESA software behaviour
* lib
+
Original ESA Java library (.jar files)
* src
+
IRAP's added Java source files
* irap-streamedds.jar
+
IRAP modified library (build from src/*java)
* build
+
Shell script to build the IRAP Java modified software
=== Configuration
You have first to update the config.properties file with your usual text editor,
to fill your EDDS username and passord
.config.properties
----
# The username that the request submitter should log on as
username = <EDDS-username>
# The password that the request submitter should log on as
password = <your-EDDS-password>
# The EDDS web server that the request submitter should submit requests to
eddsServerAddress = https://mmaasweb01.esoc.ops.esa.int:8443/edds/EddsService?wsdl
----
Check also the content of streamclient.xml file, to verify pathname of config.properties file
.streamclient.xml
----
<bean class="org.apache.camel.spring.spi.BridgePropertyPlaceholderConfigurer">
<property name="location" value="file:///home/solar/EDDS/STREAM_CLIENT.irap/config.properties"/>
</bean>
----
You can use the build script to reconstruct the IRAP Java library,
but it's not necessary if you dont't modify the corresponding java sources.
----
./build
----
Will create/update the irap-streamedds.jar library
=== User Manual
From the extraction directory, just enter :
----
# To download TM data
$ python -m EDDS TM
# To download TC data
$ python -m EDDS TC
----
[CAUTION]
====
Actually, as SWA data is not yet available, we are using PID = 76,77 filtering for TM data test purpose
====
You can have a look on EDDS.py, in create_TM_stream_request() function
----
StreamDataRequest = PktTmStream (
DataSource = "PARC",
Filter = "pids=76,77;sendRawData=true"
)
----
We will update later the EDDS.py software, to modify PID filters, for example to filter SWA data :
Filter = "pids=95-99;sendRawData=true"
You can modify also the test_TM() or test_TC() functions:
----
def test_TM ():
"""
Create a PktTmStream request, with a given duration
"""
duration = isodate.Duration (hours = 1)
req = create_TM_stream_request (duration)
job_id = send_stream_request (req)
update_properties (job_id)
run_stream_client()
----
== Acknowledgements
Please acknoledge Tom.barthe@gmail.com
* He has made available the EDDS Web services from remote python script, using SOAP protocol.
* He has also overloaded the ESA stream client software to modify the application behaviour,
in order to create a single result file per request, increasing regularly in near real-time.
