Skip to content

[Bug]: PilotSync Agent doesn't upload files to https servers #8640

Description

@Loxeris

Search before creating an issue

  • I have searched existing issues and confirmed this is not a duplicate

Bug Description

When using a web server's https address in uploadLocation of the PilotSync Agent, the pilot files are not uploaded and only file paths are provided by the PUT requests.

Steps to Reproduce

  1. run an external web server handling put requests, i.e. nginx with the following config:
    http {

        access_log /dev/stdout;
        error_log /dev/stdout;


        server {
          listen 8080;
          listen 8443 ssl;
          server_name localhost;

          ssl_certificate    hostcert.pem;
          ssl_certificate_key hostkey.pem;

          location /pilot/ {
            autoindex on;
            root /opt/dirac/webRoot/www/;
            dav_methods PUT;
            create_full_put_path  on;
            expires 1h;
            break;
          }
        }
  1. set UploadLocations = https://your-server:8443/pilot in the Systems/WorkloadManagement/Agents/PilotSyncAgent config

Expected Behavior

Pilot files are synced on the nginx server.
dirac-pilot.py, pilot.json, ... are created under /opt/dirac/webRoot/www/pilot/

Actual Behavior

A single pilot file is created at /opt/dirac/webRoot/www/pilot, its text content being the path of the file that should be uploaded last.

Environment

  • DIRAC Version: v9.1.9
  • OS: AlmaLinux

Relevant Log Output

Additional Context

Additionally PilotSync only expects 200 and 202 responses, while webDAV servers typically use 201 on creation and 204 or 200 on updates, causing unnecessary error logs.

I believe the put request should be updated with the file basename added to the url, and use file content instead of the file path in the request data.

Metadata

Metadata

Assignees

Labels

Type

Fields

No fields configured for Bug.

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions