Feature 520. Get rid of the complex pipelines and issue a single call for listing DIDs

[MytsV]

[GSoC 2025]

Fix #520

Requires merging and deployment of changes from rucio/rucio#7496 or any other PR addressing the issue.

Summary

The application has two views that require listing DIDs. The first one, DIDs List, only displays their scope and name, while loading the metadata on demand. On the other hand, during the rule creation a "deep" size of a dataset/container should be fetched, as well as its status (open).

Hence the decision to split the existing feature into two: one loads a simple list of names from the Rucio Server, the other fetches extended information with size and status.

Changes

• Made the original list DIDs features not include DIDType
• Implement an endpoint for fetching extended DID data
• Create the clean architecture feature for fetching extended DIDs
• Align the existing tests with the new expected outputs
• Cover the new gateway and feature with unit tests
• Refactor the reviewed tests to avoid duplication in stream processing
• Connect the Create Rule view to the newly added feature

Benefits

• Reduces API calls needed by WebUI 2.0, significantly decreasing latency

Followup

• Some issue was introduced to the storybooks by some earlier updates. The AGGrid doesn't become ready, making manual tests impossible. I will address it in another PR.
• Many more tests contain stream processing duplication, which should be addressed in the future.