Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Generic xds client resource watchine e2e #8183

Open
wants to merge 9 commits into
base: master
Choose a base branch
from

Conversation

purnesh42H
Copy link
Contributor

@purnesh42H purnesh42H commented Mar 19, 2025

This is the change to make generic xDS client do resource watching from xDS management server. The xDS client uses the ADS (Aggregated Data Service) stream transport channel that was created in #8144. The e2e tests are written using the listener resource type from #8144 and grpctransport from #8066.

The PR copies the existing clientimpl, authority, clientimpl_watcher from internal xdsclient code along with the helpers that are part of gRPC and then modify them to use the generic client types and interfaces.

The PR also adds String() and Equal() methods for ServerIdentifierExtension and implements them for grpctransport.ServerIdentifierExtension

The PR can be reviewed commit by commit. The first 2 commits add the String() and Equal() helpers and implementation for grpctransport respectively. The next 4 commits copy the xDS client and e2e tests files, each followed by their modification commit to use generic interfaces.

RELEASE NOTES: None

Copy link

codecov bot commented Mar 19, 2025

Codecov Report

Attention: Patch coverage is 54.85437% with 372 lines in your changes missing coverage. Please review.

Project coverage is 81.75%. Comparing base (0af5a16) to head (124b2f3).
Report is 4 commits behind head on master.

Files with missing lines Patch % Lines
xds/internal/clients/xdsclient/authority.go 47.03% 220 Missing and 21 partials ⚠️
xds/internal/clients/xdsclient/clientimpl.go 61.97% 55 Missing and 18 partials ⚠️
...ds/internal/clients/internal/testutils/wrappers.go 0.00% 24 Missing ⚠️
...s/internal/clients/grpctransport/grpc_transport.go 74.07% 10 Missing and 4 partials ⚠️
.../internal/clients/xdsclient/clientimpl_watchers.go 72.09% 9 Missing and 3 partials ⚠️
xds/internal/clients/xdsclient/xdsclient.go 46.66% 7 Missing and 1 partial ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##           master    #8183      +/-   ##
==========================================
- Coverage   82.04%   81.75%   -0.29%     
==========================================
  Files         410      416       +6     
  Lines       40251    41054     +803     
==========================================
+ Hits        33023    33563     +540     
- Misses       5867     6091     +224     
- Partials     1361     1400      +39     
Files with missing lines Coverage Δ
xds/internal/clients/internal/internal.go 97.87% <100.00%> (+8.98%) ⬆️
xds/internal/clients/internal/testutils/channel.go 100.00% <100.00%> (ø)
xds/internal/clients/xdsclient/logging.go 100.00% <100.00%> (ø)
xds/internal/clients/xdsclient/xdsconfig.go 100.00% <100.00%> (ø)
xds/internal/clients/xdsclient/xdsclient.go 55.55% <46.66%> (+55.55%) ⬆️
.../internal/clients/xdsclient/clientimpl_watchers.go 72.09% <72.09%> (ø)
...s/internal/clients/grpctransport/grpc_transport.go 82.07% <74.07%> (-8.50%) ⬇️
...ds/internal/clients/internal/testutils/wrappers.go 0.00% <0.00%> (ø)
xds/internal/clients/xdsclient/clientimpl.go 61.97% <61.97%> (ø)
xds/internal/clients/xdsclient/authority.go 47.03% <47.03%> (ø)

... and 29 files with indirect coverage changes

🚀 New features to boost your workflow:
  • Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@purnesh42H purnesh42H force-pushed the generic-xds-client-resource-watchine-e2e branch from c99ca98 to 02db583 Compare March 19, 2025 19:48
@purnesh42H purnesh42H requested a review from dfawley March 19, 2025 20:07
@purnesh42H purnesh42H added Type: Feature New features or improvements in behavior Area: xDS Includes everything xDS related, including LB policies used with xDS. labels Mar 19, 2025
@purnesh42H purnesh42H added this to the 1.72 Release milestone Mar 19, 2025
@purnesh42H
Copy link
Contributor Author

purnesh42H commented Mar 19, 2025

@dfawley could you do a pass on the implementation? For now, I have removed the metrics from client as it requires some more thought. It can be added in separate commit or PR. Otherwise the main implementation can be reviewed to make sure we are following the right approach and do the changes (if needed) before we go further. Also, I will be adding more e2e tests in subsequent commits

return ""
}
var tcParts []string
if sie.Credentials.TransportCredentials() != nil {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ideally we shouldn't need to do this. And if we do, then we should probably be doing it in the TransportCredentials type, not here.

Note that String()s are intended for human-readable formatting or for debugging, and not for exhaustive uniqueness purposes. So using it as a map key is probably a bad idea.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

So, user will have to provide String() implementation for both TransportCredentials and PerRPCCredentials?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No. Only if they want the debugging messages we would log including them to look pretty.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Modifed String() for both helper and grpctransport. PTAL

return false
}

if sie.Credentials.TransportCredentials() != nil {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is not a guaranteed correct equality check.

We should see if the credentials implements Equal and call it. If not, if they are pointers we could compare those. But if they are not, we would just have to consider them not equal and maybe log a warning or something.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Same, so user is expected provide Equal() implementation for Credentials?

Copy link
Member

@dfawley dfawley Mar 20, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

At the xdsclient level, we might want to make the extensions required to implement Equal.

At the grpctransport level, I am thinking the best we can do is be noisy if the credentials don't implement it.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Added comment on docstring for requirement of Equal in ServerIdentifierConfig.Extensions. Modified Equal() for both helper and grpctransport. PTAL.

Comment on lines 92 to 96
if sie.Credentials.PerRPCCredentials() != nil {
if sie.Credentials.PerRPCCredentials().RequireTransportSecurity() != sie2.Credentials.PerRPCCredentials().RequireTransportSecurity() {
return false
}
}
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Same comment as above, but even more strongly. RequireTransportSecurity is a boolean. If two credentials both require transport security, there is no reason to assume they are the same credential.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done as suggested above

@purnesh42H purnesh42H requested a review from dfawley March 20, 2025 20:21
@purnesh42H purnesh42H force-pushed the generic-xds-client-resource-watchine-e2e branch from 042b900 to 124b2f3 Compare March 20, 2025 20:28
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Area: xDS Includes everything xDS related, including LB policies used with xDS. Type: Feature New features or improvements in behavior
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants