We developed a set of HTTP REST APIs on top of iRODS to support users of different communities to automate both ingestion and retrieval data workflows. We built a common REST APIs layer by implementing basic functionalities, including the interaction with iRODS, within an extensible framework (RAPyDO: Rest Apis with Python on Docker) that we developed and adopted to build communities-specific REST APIs. More in details, we are collaborating with the EUropean DATa infrastructure EUDAT Collaborative Data Infrastructure (CDI); European projects like EOSC-hub and SeaDataCloud; national initiatives in collaboration with Telethon Foundation (a non-profit organization for genetic diseases research) and SIGU (Italian Society for Human Genomics). All endpoints are written by using the Python language on the Flask framework. APIs are served through an uWSGI web server deployed within a Docker container. We created a wrapper of the python irods client (PRC) to let both the core framework and communities specific APIs for easily interact with iRODS by supporting all main authentication protocols like native passwords, Pluggable Authentication Modules (PAM), Grid Security Infrastructure (GSI). To be able to support all required authentication methods we contributed to the PRC development by implementing authentication modules for both GSI and PAM. Most of iRODS-based functions that we developed can be mapped against corresponding icommands like ils, iget, iput, imv, icp, imeta, irule, iticket but also more complex functionalities have been realized, for instance streamed read/write operations from/to network sockets. To be able to execute data intensive and complex workflows, we also introduced an asynchronous layer implemented on Celery, a task management queue based on distributed message passing.
分享
引用
