Skip to content

feat: make elasticConfig.maxReplicas to a required parameter #331

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 1 commit into from
Mar 26, 2025
Merged

feat: make elasticConfig.maxReplicas to a required parameter #331

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 4 additions & 2 deletions api/inference/v1alpha1/playground_types.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -44,6 +44,7 @@ type PlaygroundSpec struct {
BackendRuntimeConfig *BackendRuntimeConfig `json:"backendRuntimeConfig,omitempty"`
// ElasticConfig defines the configuration for elastic usage,
// e.g. the max/min replicas.
// +optional
ElasticConfig *ElasticConfig `json:"elasticConfig,omitempty"`
}

Expand All @@ -56,8 +57,9 @@ type ElasticConfig struct {
MinReplicas *int32 `json:"minReplicas,omitempty"`
// MaxReplicas indicates the maximum number of inference workloads based on the traffic.
// Default to nil means there's no limit for the instance number.
// +optional
MaxReplicas *int32 `json:"maxReplicas,omitempty"`
// +kubebuilder:validation:Required
// +kubebuilder:validation:Minimum:=1
MaxReplicas int32 `json:"maxReplicas,omitempty"`
// ScaleTrigger defines the rules to scale the workloads.
// Only one trigger cloud work at a time, mostly used in Playground.
// ScaleTrigger defined here will "overwrite" the scaleTrigger in the recommendedConfig.
Expand Down
5 changes: 0 additions & 5 deletions api/inference/v1alpha1/zz_generated.deepcopy.go
View file
Open in desktop

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

3 changes: 3 additions & 0 deletions config/crd/bases/inference.llmaz.io_playgrounds.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -242,6 +242,7 @@ spec:
MaxReplicas indicates the maximum number of inference workloads based on the traffic.
Default to nil means there's no limit for the instance number.
format: int32
minimum: 1
type: integer
minReplicas:
default: 1
Expand Down Expand Up @@ -864,6 +865,8 @@ spec:
type: array
type: object
type: object
required:
- maxReplicas
type: object
modelClaim:
description: |-
Expand Down
7 changes: 1 addition & 6 deletions pkg/controller/inference/playground_controller.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -557,12 +557,7 @@ func newHPA(playground *inferenceapi.Playground) *autoscalingv2.HorizontalPodAut
}

hpa.Spec.MinReplicas = playground.Spec.ElasticConfig.MinReplicas
if playground.Spec.ElasticConfig.MaxReplicas == nil {
// The value is hardcoded, because maxReplicas is required by HPA.
hpa.Spec.MaxReplicas = 99999
} else {
hpa.Spec.MaxReplicas = *playground.Spec.ElasticConfig.MaxReplicas
}
hpa.Spec.MaxReplicas = playground.Spec.ElasticConfig.MaxReplicas

return hpa
}
4 changes: 2 additions & 2 deletions pkg/webhook/playground_webhook.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -145,8 +145,8 @@ func (w *PlaygroundWebhook) generateValidate(obj runtime.Object) field.ErrorList
allErrs = append(allErrs, field.Forbidden(specPath.Child("elasticConfig.minReplicas"), "minReplicas couldn't be 0"))
}

if playground.Spec.ElasticConfig.MinReplicas != nil && playground.Spec.ElasticConfig.MaxReplicas != nil {
if *playground.Spec.ElasticConfig.MinReplicas >= *playground.Spec.ElasticConfig.MaxReplicas {
if playground.Spec.ElasticConfig.MinReplicas != nil {
if *playground.Spec.ElasticConfig.MinReplicas >= playground.Spec.ElasticConfig.MaxReplicas {
allErrs = append(allErrs, field.Invalid(specPath.Child("elasticConfig.scaleTrigger.hpa"), *playground.Spec.ElasticConfig.MinReplicas, "minReplicas must be less than maxReplicas"))
}
}
Expand Down
2 changes: 1 addition & 1 deletion test/util/wrapper/playground.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -160,7 +160,7 @@ func (w *PlaygroundWrapper) ElasticConfig(minReplicas, maxReplicas int32) *Playg
if w.Spec.ElasticConfig == nil {
w.Spec.ElasticConfig = &inferenceapi.ElasticConfig{}
}
w.Spec.ElasticConfig.MaxReplicas = ptr.To[int32](maxReplicas)
w.Spec.ElasticConfig.MaxReplicas = maxReplicas
w.Spec.ElasticConfig.MinReplicas = ptr.To[int32](minReplicas)
return w
}
Expand Down
Loading