Exascale Astronomy vs. Cybersecurity: Unlocking French Supercomputers for the Radio Astronomy Community

How a service-based access model can unlock French national supercomputers for radio astronomy without compromising cybersecurity.

Publication · September 13, 2026 · 17 min read

Next-generation radio telescopes, such as the Square Kilometre Array (SKA) [1], will generate exascale data volumes that no local cluster can absorb. Processing them requires the parallel architectures of national high-performance computing (HPC) systems — and in France, that means the machines coordinated by GENCI [2]: Adastra at CINES [3], Jean Zay at IDRIS [4], and Joliot-Curie at TGCC [5], where the first French exascale supercomputer, Alice Recoque, is currently being installed.

The Square Kilometre Array radio telescope. Credit: SKAO.

For the radio astronomy community, the path to these machines is blocked — not by a lack of hardware, but by a systemic conflict between scientific collaboration and national security. This is the tension the work we present at the SC26 Workshops sets out to resolve: how to let a scientific community share a common application across French HPC systems without breaking the accountability rules those systems are built on.

This is not a new problem. The Low-Frequency Array (LOFAR) [6] is a European network of radio telescopes and a pathfinder to the SKA; the LOFAR as a service platform [7] demonstrated the viability of a portal-based approach for on-demand access to computing resources. It enables the radio astronomy community to process LOFAR data through dedicated pipelines deployed on distributed data and compute infrastructures co-located with the LOFAR Long-Term Archive sites, including SURFsara, the Dutch national e-infrastructure [8]. But this model was designed for grid environments and did not account for the accountability requirements imposed by the cybersecurity regulations associated with ZRR status.

Recent empirical work [9] identifies barriers to HPC usage in radio astronomy — limited accessibility, heterogeneous systems, gaps in advanced computing expertise, and increasing software complexity. Though this study focuses on commercial HPC in an Australian context, these barriers are relevant to publicly funded national infrastructures, and in the French HPC ecosystem the added cybersecurity rules imposed by the ZRR further widen the gap between radio astronomy practice and HPC operational models.

The Friction: Science vs. ZRR

In France, because supercomputers are classified as national strategic assets, the HPC centers that host them operate within the French framework for the protection of the Nation’s scientific and technical potential (PPST) [10]. Established by French regulations and described in further detail in recommendations published by ANSSI [11], this framework aims to prevent the loss, sabotage, or appropriation of sensitive knowledge and technologies by foreign powers or malicious actors. GENCI’s access rules [12] specify how these requirements are implemented in the terms and conditions governing access to national HPC resources.

To enforce the PPST, a computing center — or part of its infrastructure — is classified as a Zone à Régime Restrictif (ZRR) by ministerial order [13]. The ZRR places physical and logical access barriers around sensitive resources, and imposes a set of operational restrictions on their use:

1. Nominative access — access is granted individually to a named person, which means each HPC user must deploy, configure, and maintain their own application stack.
2. Personal responsibility — access is tied to a specific individual, so delegated or shared use of a single HPC account is incompatible with the regime.
3. Traceability — physical and logical access, and any circulation inside the ZRR, must be regulated and traced.
4. Controlled data transfers — disclosure or transfer of information to unauthorized third parties must be assessed and traceable.

On top of those regulatory constraints come the ordinary constraints of shared supercomputers: SLURM scheduling [14], execution time limits of 24 hours on Adastra, 100 hours on Jean Zay, and 72 hours on Joliot-Curie, and internet access confined to frontend nodes. Together, they make community-driven applications — a single pipeline maintained by a few experts and used by dozens of researchers — very difficult to operate on French machines.

The result is a scalability nightmare. Where a community model relies on a few experts to maintain a pipeline for dozens of researchers, the nominative rule forces every single researcher to independently deploy, configure, and maintain the entire stack on every supercomputer they use. Because each center requires its own nominative registration, the burden multiplies with every machine.

The Jean Zay supercomputer. Credit: CNRS, Cyril Frésillon.

Why Existing Solutions Don’t Fit

Several families of tools already address parts of this problem, but none of them satisfies the ZRR’s personal-responsibility requirement while still serving a whole community:

• Workflow management systems such as Pegasus [15], Nextflow [16], or Snakemake [17] orchestrate directed acyclic graphs across Cloud and HPC and can submit through a local batch scheduler. But they remain, by nature, per-user engines: each individual installs, configures, and maintains their own instance under a nominative account, and the deployment is never exposed as a shared, operated service.
• Web portals and science gateways [18], [19] hide HPC complexity behind an interface and satisfy nominative access by mapping each portal user to their own HPC account — but they do not remove the operational burden: every user still deploys and maintains the scientific application stack.
• Grid-era middleware such as GlideinWMS [20], PanDA [21], or DIRAC [22] combine pilot-based execution with Virtual Organization identities — short-lived proxy certificates issued by VOMS [23]. The incompatibility with the ZRR lies in the executing identity — a collective credential — rather than in the pull dispatch mode itself.
• Programmatic access layers such as Globus Compute (originally funcX) [24] and FirecREST [25] are the closest to our approach: they expose APIs to submit and monitor tasks on remote supercomputers. In their standard user-facing deployments, however, they authenticate users individually and associate each job with the requesting user’s HPC identity — and provide no centrally operated catalog of validated scientific applications exposed as a community service.

The ZRR requires personal responsibility and traceability for every action performed on the system. The rule that the user and the executor must be the same identity is one way to meet that requirement — and it currently prohibits an HPC account from executing jobs on behalf of a third party.

The "Globus Question": Why not use existing standards?

For many researchers, especially in the US, Globus is the gold standard for HPC data movement and identity management, and it looks like the obvious answer. So why isn't Globus used in French HPC centers subject to PPST and ZRR regulations?

The answer lies in the externalization of operational metadata.

1. The infrastructure: Globus services are hosted on Amazon Web Services and operated by Globus, while storage and computing endpoints remain operated by subscribing organizations.
2. What is stored [26]: research files are not shared with Globus, but its services store task-related information, temporary credentials, filenames, and directory paths.
3. The French mandate: any logical access to information protected by the PPST regime is strictly subject to authorization by the supervising Ministry. Externalizing that operational metadata may raise PPST and ZRR compliance concerns — including potential exposure to U.S. legal processes under 18 U.S.C. § 2713 [27]. At the time of writing, we found no publicly documented Globus Compute endpoint operated by a French national HPC center.

In short: where Globus offers convenience through cloud-based orchestration, the ZRR requires sovereign control over operational data. This is precisely why our approach does not rely on Globus: it leverages an existing framework for scientific applications as a service, made of Application Microservices (AM) for application deployment and Ephemeral Buffers (EB) for transient data logistics — providing Globus-like functionality while remaining compliant with French national security law.

The Solution: Scientific Applications as a Service

To bridge this gap, we propose decoupling the identity of the user from the identity of the executor, in a framework that exposes scientific applications as a service [28]. Instead of forcing every astronomer to become an HPC system administrator, we mutualize the use of supercomputers around a small set of precise roles:

• Users are identified by a recognized authentication federation (e.g. EduGAIN), may be transient, and need not be familiar with HPC systems.
• Maintainers are HPC experts from the scientific community who deploy scientific applications and their runtime on a target supercomputer and assume operational responsibility for the jobs executed there. Their responsibility is decoupled from development: they need not belong to the project that writes the software, but they validate every version they deploy.
• A named service account is an HPC account owned by a maintainer, through which users submit jobs without holding individual credentials. Limiting it to a single predefined scientific application simplifies the audit process for the center’s cybersecurity service.
• A user whitelist is defined for each named service account. Managed by the maintainer, it lists the users authorized to use the account’s resources, satisfying the ZRR requirement for controlled data transfers.
• A job is a submission of data for processing with fixed, pre-installed software. Once submitted, it is pulled and executed automatically by the maintainer’s runtime, which attributes execution responsibility to the maintainer and thereby satisfies the ZRR’s personal-responsibility requirement.

The required derogation is narrow: authorizing the maintainer to execute jobs that process data submitted by a third party, where that third party is identified through an authentication federation. This proposal is currently under discussion with French HPC security stakeholders.

The Architecture: How it Works

The framework is a single point of entry for launching scientific application jobs on supercomputers, without direct access to the underlying HPC centers. It wraps complex HPC workflows into a seamless service:

• Application Microservices (AM) [29] — the catalog of available scientific applications, exposed as microservices. AM orchestrates execution through job and runtime entities; because submission and execution are decoupled, users do not need a persistent connection, enabling a launch-and-forget experience for jobs that may run for several days.
• Ephemeral Buffers (EB) [30] — a transient data logistics layer providing temporary storage and a data exchange point between users and HPC systems. They are not mandatory, but they ease transfers and are accessible with only a user identity.
• The Runner — a Python script (runner.py), executed by a maintainer, that periodically polls AM for pending jobs and executes them on the target HPC system over SSH [31], [32]. This pull-based design — rather than a web service pushing jobs into the secure zone — satisfies accountability and traceability, and it enforces the user whitelist before any execution.

Scientific applications as a service architecture. Each actor operates under a distinct governance domain — users and developers under their laboratory’s policy, the API under IRISA’s, and HPC centers under their internal policy and the ZRR regulations — while maintainers sit at the intersection of the API and each center.

The Lifecycle of a Job

From the user’s perspective, a global job is a single object to watch from submission to completion:

1. Authentication — the user authenticates against the AM API and receives an authenticated client in return.
2. Application selection — the user lists the available applications, optionally filtered by name, then requests the chosen application’s contract describing its expected inputs and outputs.
3. Global job submission — based on that contract, the user prepares the job locally, describing its inputs in JSON, and submits it. The API registers it for execution.
4. Monitoring loop — the user polls the global job status after submission, without waiting for the execution to start.
5. Runner task — meanwhile the maintainer’s runner picks up the pending job, marks it running, prepares the working directory, stages the inputs, and submits a job to SLURM.
6. SLURM job — the global job and the SLURM job are distinct: the first is managed by the AM API, the second by the scheduler that runs the computation. The runner monitors the SLURM job until it ends, then collects the results.
7. Outputs retrieval — the runner writes the collected results back to the API and marks the global job completed; the user then downloads the outputs through the API, without ever connecting to the HPC system.

The four lifespans are nested: the SLURM job runs inside the runner task, which runs inside the global job, which the user monitors.

Proof of Concept: The DDF Pipeline on Jean Zay

We instantiated this architecture with the DDF Pipeline [33], [34], a community-driven radio astronomy application whose memory footprint, multi-day runtime, and reliance on external LOFAR datasets make it a demanding test case for French supercomputers under the ZRR regime.

The DDF Pipeline is a composite application integrating DDFacet [35], a direction-dependent facets imaging package, and killMS [36], a calibration package for radio interferometry. It processes Measurement Sets (MS), a standardized format for interferometer visibilities, and produces sky images in FITS format. Originally a CPU-based, shared-memory application, it requires roughly 400 GB of RAM to process LOFAR datasets: a representative run over 24 MS input files — 38.72 GB compressed — generated 594 GB of output after 68.8 hours on a single node [37].

Memory per node shapes what can run where: Adastra [38] offers 768 GB per node and is well suited to the pipeline, Jean Zay [39] has 192 GB per node and requires limiting the size of the generated sky image (GPU partitions with at least 512 GB can also be used), and Joliot-Curie [40] has 228 GB per node — with only 7 fat nodes large enough. A configuration that fits one machine may be hopeless on another.

To prepare for SKA data volumes, we co-developed a parallel version of the DDF Pipeline with partners from the ECLAT joint laboratory [41], integrating the multi-core, multi-node parallelization of DDFacet [42] with a parallel implementation of killMS. This is the version used for the results below.

The DDF Pipeline workflow — three stages, each exposed as a distinct microservice.

A complete experiment is expressed as three stages, each a specific microservice:

1. Download data — inputs are MS files given by local path or URL, a target supercomputer, and its identifier. The microservice checks compatibility with the supercomputer by verifying that the URL domain appears on the user whitelist assigned to the named service account; local paths are staged through ebuffers. The output is a global job identifier.
2. Run DDF Pipeline — inputs are a configuration file or specific options (e.g. image size) and the global job identifier. The microservice validates the configuration and access to the input data, prepares the execution directory, and submits a job to SLURM, returning the global job identifier so the user can poll its status.
3. Upload results — inputs are the destination (local path or URL) and the global job identifier. The microservice uploads the data using ebuffers for a local path or via the API for a remote repository.

The Results

The workflow was exercised on the Jean Zay supercomputer to validate the architecture. We measured three things: where the data can live, how the pipeline scales, and what the service layer costs.

• Data access — for the LOFAR LoTSS L234026 8-hour integration dataset, SURFsara delivered the highest average download throughput (137.9 MB/s, though with the largest variability) ahead of EOSC [43] (51.85 MB/s) and CNRS Drive (52.36 MB/s). Upload favored EOSC (28.73 MB/s, coefficient of variation 34.3%) over CNRS Drive (13.81 MB/s, 84.7%), making SURFsara and EOSC the viable endpoints for this experiment.
• Scalability — elapsed time fell from 68.8 hours on 1 node to 8.7 hours on 24 nodes (≈7.9× speedup), passing through 14.4 hours on 6 nodes and 10.2 hours on 12 nodes. Average CPU utilization decreased from 63.5% to 31%, primarily because unparallelized DDFacet deconvolution steps limit scaling efficiency — parallelizing them is ongoing work.
• Service overhead — job submission through the API added only 0.25 to 0.96 seconds for configurations of 0 (baseline), 1, and 5 ebuffers. This is negligible against workflow executions that range from several hours to several days.

Job submission sequence diagram. The four lifespans are nested: the SLURM job (gray) runs inside the runner task (yellow), which runs inside the global job (blue) monitored by the user (purple).

The Road Ahead: Beyond the Proof of Concept

The central limitation of this work is not technical but regulatory: the proposed decoupling requires a formal derogation from an explicit ZRR prohibition, and its acceptance by French HPC security authorities remains the next milestone toward full deployment.

Operating the DDF Pipeline as a service on national infrastructure also surfaced three practical consequences:

• Quality of service — SLURM prioritizes accounts with low recent usage. With all users accessing resources through a single account, this creates significant allocation delays, so the fair-share policy for the named service account will need to be adapted.
• Storage — a single run generates hundreds of gigabytes, while supercomputer quotas are limited. Results must be transferred to an external source and working directories cleared after each run, which also limits how many jobs can run in parallel and motivates distributing jobs across multiple HPC systems.
• Reproducibility — step-level reproducibility is impractical for pipelines generating hundreds of gigabytes of intermediate products per run. We instead target full-workflow reproducibility, preserving input references, execution parameters, and environment specifications.

Cross-center executions of the DDF Pipeline between Jean Zay and Adastra have been validated, motivating future integration of cross-facility deployment [44] within the EB/AM framework. The named service account model remains under discussion with French HPC stakeholders, and future work will extend deployment to additional French supercomputers — Adastra and Joliot-Curie.

Conclusion

Security should be an enabler of science, not a barrier. By shifting from a "user-access" model to a "service-access" model — and by separating the user’s identity from the executor’s identity — we can reconcile the ZRR’s accountability requirements with the way scientific communities actually work, and open French supercomputers to the radio astronomy community that needs them most.

References

1. P. E. Dewdney, P. J. Hall, R. T. Schilizzi, and T. J. L. W. Lazio, “The Square Kilometre Array,” Proc. IEEE, vol. 97, no. 8, pp. 1482–1496, Aug. 2009.
2. GENCI, “Grand Equipement National de Calcul Intensif (GENCI).”
3. CINES, “Centre Informatique National de l’Enseignement Supérieur (CINES).”
4. CNRS, “Institut du Développement et des Ressources en Informatique Scientifique (IDRIS).”
5. CEA, “Très Grand Centre de Calcul (TGCC).”
6. M. P. van Haarlem, M. W. Wise, A. W. Gunst, G. Heald, J. P. McKean, J. W. T. Hessels, et al., “LOFAR: The LOw-Frequency ARray,” Astron. Astrophys., vol. 556, p. A2, Aug. 2013.
7. A. P. Mechev, J. B. R. Oonk, A. Plaat, A. Danezi, and T. W. Shimwell, “Building LOFAR as a Service,” in Proc. Astronomical Data Anal. Softw. Syst. XXVIII (ADASS), vol. 523, San Francisco, CA, USA: Astronomical Society of the Pacific, 2019, p. 677.
8. J. Templon and J. Bot, “The Dutch National e-Infrastructure,” in Proc. Int. Symp. Grids Clouds (ISGC 2016), vol. 270, Taipei, Taiwan: SISSA Medialab, Jan. 2017, p. 020.
9. I. P. Kemp, S. J. Tingay, S. D. Midgely, and D. A. Mitchell, “Experiences of commercial supercomputing in radio astronomy data processing,” Astron. Comput., vol. 54, p. 101013, Jan. 2026.
10. Légifrance, “Protection du Potentiel Scientifique et Technique (PPST),” Décret n° 2024-430 du 14 mai 2024.
11. ANSSI, “Protection du Potentiel Scientifique et Technique de la Nation (PPST),” Agence nationale de la sécurité des systèmes d’information.
12. GENCI, “Modalités d’accès aux ressources nationales,” Réf. CD230106, version 1.15, May 2023.
13. Légifrance, “Zones à Régime Restrictif (ZRR),” Code pénal, Art. R413-5-1, 2025.
14. A. B. Yoo, M. A. Jette, and M. Grondona, “SLURM: Simple Linux Utility for Resource Management,” in Proc. Workshop Job Scheduling Strategies for Parallel Processing (JSSPP), vol. 2862, Berlin, Germany: Springer, Jun. 2003, pp. 44–60.
15. E. Deelman et al., “The Evolution of the Pegasus Workflow Management Software,” Comput. Sci. Eng., vol. 21, no. 4, pp. 22–36, Jul. 2019.
16. P. Di Tommaso, M. Chatzou, E. W. Floden, P. P. Barja, E. Palumbo, and C. Notredame, “Nextflow enables reproducible computational workflows,” Nat. Biotechnol., vol. 35, no. 4, pp. 316–319, Apr. 2017.
17. J. Köster and S. Rahmann, “Snakemake—a scalable bioinformatics workflow engine,” Bioinformatics, vol. 28, no. 19, pp. 2520–2522, Oct. 2012.
18. P. Calegari, M. Levrier, and P. Balczynski, “Web Portals for High-performance Computing: A Survey,” ACM Trans. Web, vol. 13, no. 1, pp. 5:1–5:36, Feb. 2019.
19. M. Barker et al., “The global impact of science gateways, virtual research environments and virtual laboratories,” Future Generation Computer Systems, vol. 95, pp. 240–248, Jun. 2019.
20. I. Sfiligoi, “glideinWMS—a generic pilot-based workload management system,” J. Phys.: Conf. Ser., vol. 119, no. 6, p. 062044, Jul. 2008.
21. T. Maeno, “PanDA: distributed production and distributed analysis system for ATLAS,” J. Phys.: Conf. Ser., vol. 119, no. 6, p. 062036, Jul. 2008.
22. A. Casajus, R. Graciani, S. Paterson, A. Tsaregorodtsev, and LHCb DIRAC Collaboration, “DIRAC pilot framework and the DIRAC Workload Management System,” J. Phys.: Conf. Ser., vol. 219, no. 6, p. 062049, Apr. 2010.
23. R. Alfieri et al., “From gridmap-file to VOMS: managing authorization in a Grid environment,” Future Gener. Comput. Syst., vol. 21, no. 4, pp. 549–558, Apr. 2005.
24. R. Chard et al., “funcX: A Federated Function Serving Fabric for Science,” in Proc. 29th Int. Symp. High-Perform. Parallel Distrib. Comput. (HPDC ’20), Stockholm, Sweden, Jun. 2020, pp. 65–76.
25. F. A. Cruz et al., “FirecREST: a RESTful API to HPC systems,” in Proc. IEEE/ACM Int. Workshop Interoperability Supercomput. Cloud Technol. (SuperCompCloud), Nov. 2020, pp. 21–26.
26. Globus, “High Assurance Security Overview,” Globus documentation, University of Chicago.
27. U.S. Code, “Required preservation and disclosure of communications and records,” 18 U.S.C. 2713, Legal Information Institute, Cornell Law School, Mar. 2018.
28. L. Morin, F. Bodin, and G. Nyastikor, “HPC Software as a Service: A Flexible Approach to Data Logistics,” presented at the 3rd Int. Workshop on Scalable Compute Continuum (WSCC 2025), Dresden, Germany, Aug. 2025.
29. L. Morin and G. Nyastikor, “Application Microservices (AM),” Online API documentation.
30. L. Morin and G. Nyastikor, “Ephemeral Buffers (EB),” Online API documentation.
31. CINES, “Adastra supercomputer user documentation.”
32. IDRIS, “SSH access to the Jean Zay supercomputer.”
33. C. Tasse et al., “The LOFAR Two-meter Sky Survey: Deep Fields Data Release 1 - I. Direction-dependent calibration and imaging,” Astron. Astrophys., vol. 648, p. A1, Apr. 2021.
34. M. Hardcastle, T. Shimwell, C. Tasse, and W. Williams, “DDF Pipeline source code,” GitHub repository.
35. C. Tasse, “DDFacet source code: A facet-based radio imaging package,” GitHub repository.
36. C. Tasse, “killMS source code: A Wirtinger-based direction-dependent radio interferometric calibration package,” GitHub repository.
37. M. Certenais, F. Bodin, and L. Morin, “A description of the radio astronomy data processing tool DDF Pipeline,” arXiv:2509.03075, Sep. 2025.
38. CINES, “Adastra supercomputer architecture.”
39. IDRIS-CNRS, “Jean Zay supercomputer architecture.”
40. TGCC-CEA, “TGCC Public Documentation,” Sep. 2026.
41. ECLAT, “Extreme Computing Lab for Astronomical Telescopes (ECLAT).”
42. N. Monnier et al., “Multi-core multi-node parallelization of the radio interferometric imaging pipeline DDFacet,” in Proc. IEEE Workshop Signal Process. Syst. (SiPS), Rennes, France, Nov. 2022, pp. 1–6.
43. P. Budroni, J.-C. Burgelman, and M. Schouppe, “Architectures of Knowledge: The European Open Science Cloud,” ABI Technik, vol. 39, no. 2, pp. 130–141, Jul. 2019.
44. K. B. Antypas et al., “Enabling discovery data science through cross-facility workflows,” in Proc. IEEE Int. Conf. Big Data (Big Data), Orlando, FL, USA, Dec. 2021, pp. 3671–3680.

Keep reading

• All entries
• The research
• Get in touch