Bug: Application crash when attempting to renew ODP subscription, Microsoft Graph API responds with 308 Permanent Redirect without Location Header

[sebastian-be]

Describe the bug

Seems to be crashing on webhook data.
With webhooks="false" no crashes

Operating System Details

Linux vault 6.8.0-52-generic #53-Ubuntu SMP PREEMPT_DYNAMIC Sat Jan 11 00:06:25 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux

Distributor ID: Ubuntu
Description:    Ubuntu 24.04.2 LTS
Release:        24.04
Codename:       noble

Client Installation Method

From Source

OneDrive Account Type

Personal

What is your OneDrive Application Version

onedrive v2.5.5-2-g62c3d87

What is your OneDrive Application Configuration

./onedrive-v255 --display-config
Reading configuration file: /home/user/.config/onedrive/config
Configuration file successfully loaded
Disabling GUI notifications as per user configuration
Application version                          = onedrive v2.5.5-2-g62c3d87
Compiled with                                = LDC 2106
Curl version                                 = libcurl/8.12.0 OpenSSL/3.0.13 zlib/1.3 brotli/1.1.0 zstd/1.5.5 libidn2/2.3.7 libpsl/0.21.2 nghttp2/1.59.0 librtmp/2.3 OpenLDAP/2.6.7
User Application Config path                 = /home/user/.config/onedrive
System Application Config path               = /etc/onedrive
Applicable Application 'config' location     = /home/user/.config/onedrive/config
Configuration file found in config location  = true - using 'config' file values to override application defaults
Applicable 'sync_list' location              = /home/user/.config/onedrive/sync_list
Applicable 'items.sqlite3' location          = /home/user/.config/onedrive/items.sqlite3
Config option 'drive_id'                     = 
Config option 'sync_dir'                     = /homext/user/onedrive
Config option 'enable_logging'               = false
Config option 'log_dir'                      = /var/log/onedrive
Config option 'disable_notifications'        = true
Config option 'skip_dir'                     = 
Config option 'skip_dir_strict_match'        = false
Config option 'skip_file'                    = ~*|.~*|*.tmp
Config option 'skip_dotfiles'                = false
Config option 'skip_symlinks'                = false
Config option 'monitor_interval'             = 1800
Config option 'monitor_log_frequency'        = 12
Config option 'monitor_fullscan_frequency'   = 12
Config option 'read_only_auth_scope'         = false
Config option 'dry_run'                      = false
Config option 'upload_only'                  = false
Config option 'download_only'                = false
Config option 'local_first'                  = false
Config option 'check_nosync'                 = false
Config option 'check_nomount'                = false
Config option 'resync'                       = false
Config option 'resync_auth'                  = false
Config option 'cleanup_local_files'          = false
Config option 'disable_permission_set'       = false
Config option 'transfer_order'               = default
Config option 'classify_as_big_delete'       = 1000
Config option 'disable_upload_validation'    = false
Config option 'disable_download_validation'  = false
Config option 'bypass_data_preservation'     = false
Config option 'no_remote_delete'             = false
Config option 'remove_source_files'          = false
Config option 'sync_dir_permissions'         = 755
Config option 'sync_file_permissions'        = 644
Config option 'space_reservation'            = 52428800
Config option 'permanent_delete'             = false
Config option 'write_xattr_data'             = false
Config option 'application_id'               = d50ca740-c83f-4d1b-b616-12c519384f0c
Config option 'azure_ad_endpoint'            = 
Config option 'azure_tenant_id'              = 
Config option 'user_agent'                   = ISV|abraunegg|OneDrive Client for Linux/v2.5.5-2-g62c3d87
Config option 'force_http_11'                = false
Config option 'debug_https'                  = false
Config option 'rate_limit'                   = 0
Config option 'operation_timeout'            = 3600
Config option 'dns_timeout'                  = 60
Config option 'connect_timeout'              = 10
Config option 'data_timeout'                 = 60
Config option 'ip_protocol_version'          = 1
Config option 'threads'                      = 8
Config option 'max_curl_idle'                = 120
Compile time option --enable-notifications   = false

Selective sync 'sync_list' configured        = false

Config option 'sync_business_shared_items'   = false

Config option 'webhook_enabled'              = true
Config option 'webhook_public_url'           = https://host.duckdns.org:8080/webhooks/onedrive
Config option 'webhook_listening_host'       = 127.0.0.1
Config option 'webhook_listening_port'       = 8080
Config option 'webhook_expiration_interval'  = 300
Config option 'webhook_renewal_interval'     = 300
Config option 'webhook_retry_interval'       = 60

What is your 'curl' version

curl 8.12.0 (x86_64-pc-linux-gnu) libcurl/8.12.0 OpenSSL/3.0.13 zlib/1.3 brotli/1.1.0 zstd/1.5.5 libidn2/2.3.7 libpsl/0.21.2 nghttp2/1.59.0 librtmp/2.3 OpenLDAP/2.6.7
Release-Date: 2025-02-05
Protocols: dict file ftp ftps gopher gophers http https imap imaps ipfs ipns ldap ldaps mqtt pop3 pop3s rtmp rtsp smb smbs smtp smtps telnet tftp ws wss
Features: alt-svc AsynchDNS brotli HSTS HTTP2 HTTPS-proxy IDN IPv6 Largefile libz NTLM PSL SSL threadsafe TLS-SRP UnixSockets zstd

Where is your 'sync_dir' located

Local

What are all your system 'mount points'

LABEL=main      /               btrfs   commit=5,compress=lzo,subvol=@ 0 1
UUID=9F5D-9055  /boot/efi       vfat    noauto,defaults        0       1
LABEL=main      /home           btrfs   commit=5,subvol=@home 0 2
LABEL=store     /homext         btrfs   commit=5,subvol=@homext,nodatacow 0 2
LABEL=swap      none            swap    sw              0       0

What are all your local file system partition types

NAME   FSTYPE FSVER LABEL UUID                                 FSAVAIL FSUSE% MOUNTPOINTS
sda                                                                           
├─sda1 vfat   FAT32       9F5D-9055                                           
├─sda2 swap   1     swap  c6687b6c-2536-4dce-9c27-80c4558b5177                [SWAP]
└─sda3 btrfs        main  7aa78bc2-2cca-43b1-b68a-bf5edb14b592   21.3G    23% /mnt/main
                                                                              /home
                                                                              /
sdb    btrfs        store 38970aad-fac8-4ca6-9632-36d9eeb7dd9d    1.9T    61% /homext

How do you use 'onedrive'

sharing content using linux client, microsoft client (native app) & Android client (native app)

Steps to reproduce the behaviour

configure and use webhooks

Complete Verbose Log Output

./onedrive-v255 --monitor -v        
Reading configuration file: /home/user/.config/onedrive/config
Configuration file successfully loaded
Using 'user' configuration path for application config and state data: /home/user/.config/onedrive
Disabling GUI notifications as per user configuration
Forcing client to use IPv4 connections only
Attempting to contact Microsoft OneDrive Login Service
Successfully reached Microsoft OneDrive Login Service
Checking Application Version ...
Attempting to initialise the OneDrive API ...
Configuring Global Azure AD Endpoints
The OneDrive API was initialised successfully
Opening the item database ...
Application Version:   onedrive v2.5.5-2-g62c3d87
Account Type:          personal
Default Drive ID:      xxxxxxxxxxxxxxxx
Default Root ID:       xxxxxxxxxxxxxxxx!sea8cc6beffdb43d7976fbc7da445c639
Microsoft Data Centre: West Europe
Remaining Free Space:  1028.90 GB (1104772962713 bytes)
Sync Engine Initialised with new Onedrive API instance
All application operations will be performed in the configured local 'sync_dir' directory: /homext/user/onedrive
OneDrive synchronisation interval (seconds): 1800
Maximum allowed open files:                  9223372036854775807
Maximum allowed inotify user watches:        123011
Initialising filesystem inotify monitoring ...
Monitoring directory: .
Monitoring directory: ./backup
Monitoring directory: ./Pictures
Monitoring directory: ./Pictures/Camera Roll
Monitoring directory: ./Pictures/Samsung Gallery
Monitoring directory: ./.hidden
Monitoring directory: ./Documents
Monitoring directory: ./Bijlagen
Performing initial synchronisation to ensure consistent local state ...
Started OneDrive API Webhook server
Initialising webhook subscription for updates ...
OneDrive API Webhook: handled validation request
Created new subscription 15202c07-46e0-4624-8d38-5db3b283bea7 with expiration: 2025-03-22T19:35:14.285Z
Attempting to contact Microsoft OneDrive Login Service
Successfully reached Microsoft OneDrive Login Service
Starting a sync with Microsoft OneDrive
Fetching /delta response from the OneDrive API for Drive ID: xxxxxxxxxxxxxxxx
Processing API Response Bundle: 1 - Quantity of 'changes|items' in this bundle to process: 0
Finished processing /delta JSON response from the OneDrive API
No changes or items that can be applied were discovered while processing the data received from Microsoft OneDrive
Performing a database consistency and integrity check on locally stored data
Processing DB entries for this Drive ID: xxxxxxxxxxxxxxxx
Processing: /homext/user/onedrive
The directory has not changed
Processing: Documents
The directory has not changed
Processing: Getting started with OneDrive.pdf
The file has not changed
Processing: Pictures
The directory has not changed
...
The file has not changed
Processing: backup
The directory has not changed
Processing: .hidden
The directory has not changed
Scanning the local file system '/homext/user/onedrive' for new data to upload
Performing a last examination of the most recent online data within Microsoft OneDrive to complete the reconciliation process
Fetching /delta response from the OneDrive API for Drive ID: xxxxxxxxxxxxxxxx
Processing API Response Bundle: 1 - Quantity of 'changes|items' in this bundle to process: 0
Finished processing /delta JSON response from the OneDrive API
No changes or items that can be applied were discovered while processing the data received from Microsoft OneDrive
Sync with Microsoft OneDrive is complete
Renewing webhook subscription for updates ...
OneDrive API Webhook: handled validation request
Stopped OneDrive API Webhook server
Deleted subscription
Monitored directory removed: .hidden/
Monitored directory removed: Documents/
Monitored directory removed: backup/
Monitored directory removed: Pictures/
Monitored directory removed: ./
Monitored directory removed: Bijlagen/
Monitored directory removed: Pictures/Samsung Gallery/
Monitored directory removed: Pictures/Camera Roll/
Attempting to perform a database vacuum to optimise database
Database vacuum is complete
std.json.JSONException@std/json.d(317): JSONValue is not an object
----------------
??:? pure @safe noreturn std.exception.bailOut!(std.json.JSONException).bailOut(immutable(char)[], ulong, scope const(char)[]) [0x761d4a2e9744]
??:? inout pure ref @safe inout(std.json.JSONValue) std.json.JSONValue.opIndex(return scope immutable(char)[]) [0x761d4a273583]
src/webhook.d:251 [0x576a306fb838]
src/webhook.d:132 [0x576a306fb4ad]
src/main.d:1006 [0x576a30773181]
??:? void rt.dmain2._d_run_main2(char[][], ulong, extern (C) int function(char[][])*).runAll() [0x761d4a5cddcc]
??:? _d_run_main2 [0x761d4a5cdbe6]
??:? _d_run_main [0x761d4a5cda3c]
/usr/lib/ldc/x86_64-linux-gnu/include/d/core/internal/entrypoint.d:42 [0x576a3079f6f1]
??:? [0x761d49c2a1c9]
??:? __libc_start_main [0x761d49c2a28a]
??:? [0x576a30648194]

Screenshots

na

Other Log Information or Details

na

Additional context

No response

[abraunegg]

@sebastian-be
Please can you provide a full verbose debug log as per https://github.com/abraunegg/onedrive/wiki/Generate-debug-log-for-support

[abraunegg]

@sebastian-be
The reason for the error is that the error message that is being passed in, the code is expecting to be a JSON .. which, potentially in your case it is not.

A full debug log will be needed to sort this issue out.

Please provide this verbose debug log ASAP.

[sebastian-be]

Email with debuglog sent to support@mynas.com.au.

[abraunegg]

@sebastian-be
Many thanks for the debug log.

As this issue relates to:

1. Webhooks
2. Using a Microsoft Personal Account impacted by the 'sea8cc6beffdb43d7976fbc7da445c639' backend Microsoft change
3. Using a Microsoft Data Centre in Europe
4. The Microsoft Graph API is responding with a 308 error code detailing a 'Permanent Redirect'

I have 100% zero way to test any fix to validate that this is resolved - sorry.

Please can you test the PR below to potentially resolve your issue.

First install all the require platform dependencies to build the client on your respective platforms. Please read https://github.com/abraunegg/onedrive/blob/master/docs/install.md#building-from-source---high-level-requirements and then follow correctly for your platform.

Once this is done, to clone the PR to resolve your issue, you can use a script like the following:

#!/bin/bash

PR=3172

rm -rf ./onedrive-pr${PR}
git clone https://github.com/abraunegg/onedrive.git onedrive-pr${PR}
cd onedrive-pr${PR}
git fetch origin pull/${PR}/head:pr${PR}
git checkout pr${PR}

# Configure and Build
./configure --enable-debug --enable-notifications; make clean; make;
./onedrive --version

This script will create a local folder called onedrive-pr3172 with the PR version.

To run the PR, you need to run the client from the PR build directory:

./onedrive <any other options needed>

To install the PR, you will need to perform sudo make install to install the PR version to your system.

When running the PR, your version should be: onedrive v2.5.5-3-gbf1bdef or greater.

[abraunegg]

@sebastian-be
Many thanks for the latest debug log via email.

Drilling into the details, Microsoft has broken their API yet again, now in this new area. This change has been brought about because of your Microsoft Account Change to the new Microsoft OneDrive backend for Personal Accounts. You can see this because your OneDrive 'root' now contains the string sea8cc6beffdb43d7976fbc7da445c639

What is Broken

RFC 7538 specifies HTTP 308 Permanent Redirect. This RFC introduces the 308 status code, indicating that the requested resource has been permanently moved to a new URI. Clients should use the new URI for future requests, and the request method and body should remain unchanged.

Microsoft Graph API fails to send any Location: header information - you can review this in the debug log by looking for DEBUG: HTTP Response Headers: thus the client has zero capability to redirect to the new location as requested by the 308 response:

DEBUG: OneDrive API Webhook: handled validation request
< HTTP/2 308 
< cache-control: no-cache
< strict-transport-security: max-age=31536000
< request-id: 56afacb6-e914-4832-bade-b75e67a6fa9c
< client-request-id: 56afacb6-e914-4832-bade-b75e67a6fa9c
< x-ms-ags-diagnostic: {"ServerInfo":{"DataCenter":"West Europe","Slice":"E","Ring":"5","ScaleUnit":"005","RoleInstance":"AM4PEPF0002AB4C"}}
< date: Tue, 25 Mar 2025 09:16:27 GMT
< content-length: 0
< 
* Connection #0 to host graph.microsoft.com left intact
DEBUG: HTTP Response Headers: ["strict-transport-security":"max-age=31536000", "x-ms-ags-diagnostic":"{\"ServerInfo\":{\"DataCenter\":\"West Europe\",\"Slice\":\"E\",\"Ring\":\"5\",\"ScaleUnit\":\"005\",\"RoleInstance\":\"AM4PEPF0002AB4C\"}}", "date":"Tue, 25 Mar 2025 09:16:27 GMT", "client-request-id":"56afacb6-e914-4832-bade-b75e67a6fa9c", "content-length":"0", "request-id":"56afacb6-e914-4832-bade-b75e67a6fa9c", "cache-control":"no-cache"]
DEBUG: HTTP Status Line: 308  (2.0)
DEBUG: CurlEngine cleanup() called on instance id: 5kG8LIg6QvRr7WUK
DEBUG: Microsoft Graph API Response: 
DEBUG: Handling a OneDrive API exception:
DEBUG: Curl debugging: 
< patch https://graph.microsoft.com/v1.0/subscriptions/920c9624-678f-4d40-9ebd-24bc067a5e90
< Content-Type: application/json

----
{"expirationDateTime":"2025-03-25T09:21:27.2788768Z"}
----
< 308  (2.0)
> strict-transport-security: max-age=31536000
> x-ms-ags-diagnostic: {"ServerInfo":{"DataCenter":"West Europe","Slice":"E","Ring":"5","ScaleUnit":"005","RoleInstance":"AM4PEPF0002AB4C"}}
> date: Tue, 25 Mar 2025 09:16:27 GMT
> client-request-id: 56afacb6-e914-4832-bade-b75e67a6fa9c
> content-length: 0
> request-id: 56afacb6-e914-4832-bade-b75e67a6fa9c
> cache-control: no-cache
Curl response: 

This behavior is unconventional, as the 308 status code typically includes a Location header to indicate the new URI. Such cases suggest potential misconfigurations or unexpected behaviors within the API.

What is supposed to happen

308 Permanent Redirect is supposed to include a Location: header. It tells the client to repeat the same request at a new URL. However, the responses you’re getting do not include a Location header

This is misuse of the 308 status code by the Microsoft Graph API in this context. Based on Microsoft documentation, 308 is not a standard response from PATCH operations. It might be an internal load-balancer issue or a rare bug in the West Europe Graph endpoint.

If the subscription cannot be renewed, Microsoft Graph is supposed to return a 4xx or 5xx code, such as 400 Bad Request or 404 Not Found — not a 308.

Next Steps

Please raise a bug report with Microsoft for their broken platform - this is something which I cannot fix within the application code.

I have raised OneDrive/onedrive-api-docs#1895 to at least have some visibility elsewhere

@nathan-gs
Please can you see if you can raise this issue with the correct team - yet another example of a broken platform

@patrick-rodgers , @rgregg-msft , @chackman , @rgregg , @ificator
Yet another example of how the new Microsoft OneDrive Personal backend platform is broken

[abraunegg]

@sebastian-be
The fix for the initial crash will be merged in, but your actual issue is not one that can be solved with this client any further.

[sebastian-be]

Thx you for looking into this, that made the client-side gracefully handle the issue.