0.80.1

The 0.80.1 release focuses on bug fixes and performance improvements following the major 0.80.0 update. This release addresses several critical issues, particularly improving the CLI functionality when used with the REST API through a deployed ZenML instance. Additionally, this version introduces a restructured documentation architecture for improved user experience.

Improvements

• Import integrations lazily for better performance
• Added ability to store a default project for users

Fixes

• Fixed CLI combined with RestZenStore and filters with multiple entries
• Fixed stack validation for incluster Kubernetes orchestrator
• Fixed stack and component URL when connected to a cloud workspace
• Fixed code repository host fallback
• Fixed version validation
• Various other minor bugfixes

Documentation

• Restructured entire documentation for better organization
• Fixed broken links in API documentation
• Refined logging to debug level for service connectors
• Removed redundant log messages

What's Changed

• Adding 0.80.0 to the migration tests by @bcdurak in #3442
• Adding the disabled flavor test back by @bcdurak in #3431
• Stop CLI profiler running so much by @strickvl in #3449
• Add missing fallback host for code repositories by @schustmi in #3434
• Fix stack and component URL when connected to a cloud workspace by @schustmi in #3451
• Fix stack validation for incluster Kubernetes orchestrator by @schustmi in #3450
• Bump click dependency by @strickvl in #3445
• Fix 0.80.0 database migration by @stefannica in #3453
• Pin the ZenML Terraform provider version by @stefannica in #3443
• Import integrations lazily by @stefannica in #3419
• Remove Segment analytics script and scarf image load. by @htahir1 in #3455
• Restructure entire docs by @htahir1 in #3447
• Refactor logging to debug level for service connectors by @htahir1 in #3456
• Removing redundant log messages by @bcdurak in #3459
• Fix broken link in API documentation table by @htahir1 in #3462
• Add the ability to store a default project for a user by @schustmi in #3457
• Fixing the CLI combined with RestZenStore and filters with multiple entries by @bcdurak in #3464
• Don't ask for active project when listing projects by @stefannica in #3466
• Use the build python version to collect stack requirements for run templates by @stefannica in #3465
• Fix version validation by @bcdurak in #3467

Full Changelog: 0.80.0...0.80.1

0.80.0

The 0.80.0 release is one of our biggest updates in a while! This version introduces a major refactoring of workspaces into projects, enhances tagging capabilities, and improves GitLab repository support. This release also features significant performance optimizations for Docker builds and CLI operations.

Features

• For our Pro users: Refactored workspaces into projects with improved RBAC API resource format (We will release separate docs on this soon.)
• Enhanced tagging system with resource type filtering and exclusive tag behavior
• Added persistent resource support for the Vertex orchestrator
• Store build duration information for better tracking
• Allow passing step artifacts to specify upstream steps
• Support for environment variables in KubernetesPodSettings

Improvements

• Updated devel dockerfiles to make rebuilds faster
• Improved CLI response time through optimized imports
• Allow registering public GitLab repositories without token
• Enable Weave integration in Wandb settings
• Allow the service account project ID to be overridden in the GCP service connector
• Pass API token as Kubernetes secret, allowing Kubernetess orchestrator to run workloads without exposing any sensitive API tokens in the environment

Fixes

• Fixed GitLab URL parsing and matching
• Corrected CLI command to describe flavors
• Fixed taggable filter model and filter models with multiple inputs
• Fixed project statistics endpoint and ZenML Pro project URLs
• Fixed the ACR support in the Azure service connector
• Resolved SkyPilot Orchestrator cluster name handling
• Fixed deprecation messages for GitHub code repository
• Don't retry REST API calls if runtime errors occur

Documentation

• Renamed API Docs to SDK Docs for clarity
• Fixed SDK docs rendering with proper directory structure and links
• Removed deprecated caveat from Kubernetes docs
• Various documentation fixes and clarifications

What's Changed

• Update devel dockerfiles to make rebuilds faster by @stefannica in #3385
• Deepchecks fix for the CI by @bcdurak in #3389
• Fixing the CI by @bcdurak in #3391
• Fixing the zenml login hint for separated names by @bcdurak in #3388
• bugfix: correctly parse and match Gitlab URLs by @dragosmc in #3392
• bugfix: pass iterator to gitlab by @dragosmc in #3393
• Allow registering public gitlab repositories without token by @schustmi in #3394
• Fix CLI command to describe flavors by @schustmi in #3390
• Store build duration by @schustmi in #3386
• Removed deprecated caveat from kubernetes docs by @AlexejPenner in #3395
• fix doc confusion by @VicSev in #3397
• Improved tagging by @bcdurak in #3360
• Refactor workspaces into projects by @stefannica in #3364
• Fix taggable filter model by @schustmi in #3403
• Testing the CLI with the profiler by @bcdurak in #3400
• Don't retry REST API calls if runtime errors occur by @stefannica in #3408
• Allow the service account project ID to be overridden in the GCP service connector by @stefannica in #3398
• Allow passing step artifacts to specify upstream steps by @schustmi in #3401
• Add Reo Javascript snippet to main.html by @htahir1 in #3409
• Rename workspace to project by @stefannica in #3407
• Enable Weave integration in Wandb settings by @htahir1 in #3359
• Add persistent resource support for the vertex orchestrator by @schustmi in #3396
• Minor fix for the docs by @avishniakov in #3411
• Listing tags filtered by resource type by @bcdurak in #3406
• Adding removing tags with various update models by @bcdurak in #3404
• Exclusive tag behavior by @bcdurak in #3405
• Rename tenant to workspace and implement new RBAC API resource format by @stefannica in #3414
• API Docs -> SDK Docs by @htahir1 in #3415
• Removed step by @AlexejPenner in #3416
• Fix unbound variable access by @schustmi in #3412
• Allow setting environment variables through KubernetesPodSettings by @schustmi in #3413
• Fix SDK docs rendering with proper directory structure and links by @strickvl in #3374
• Fix deprecation message for github code repository by @schustmi in #3418
• Fix project statistics endpoint by @schustmi in #3420
• Track project creation in onboarding state by @schustmi in #3423
• Fix the ACR support in the Azure service connector by @stefannica in #3424
• Limiting the mlflow dependency by @bcdurak in #3422
• Pass API token as kubernetes secret by @schustmi in #3421
• Improve the CLI response time through imports by @bcdurak in #3399
• Fetch model hydrated during deletion process by @schustmi in #3427
• Fix ZenML Pro project URLs for pipeline runs and model versions by @stefannica in #3426
• Add missing functions and classes to root init exports by @schustmi in #3428
• Fix doc links and comment test out by @htahir1 in #3430
• fix: SkypilotBaseOrchestrator handle given cluster_name and correct reuse by @BjoernBiltzinger in #3417
• Upgrading the skypilot dependency by @bcdurak in #3429
• Fixing filter models with multiple inputs by @bcdurak in #3410
• Add project usage tracking by @schustmi in #3435

New Contributors

• @dragosmc made their first contribution in #3392
• @VicSev made their first contribution in #3397
• @BjoernBiltzinger made their first contribution in #3417

Full Changelog: 0.75.0...0.80.0

0.75.0

The 0.75.0 release introduces dashboard enhancements for stack component management along with improvements to documentation and service connector capabilities. Users can now create and update stack components directly from the dashboard.

Features

• Create and update stack components directly from the dashboard
• Custom authentication method support during auto-configuration of service connectors
• Enhanced model artifact retrieval by creation date instead of version name
• Additional SageMaker environment settings

Improvements

• Expanded fastapi dependency range for better compatibility
• Improved pipeline source root documentation and logging
• Better sorting functionality when using custom fetching

Fixes

• Fixed registration of components with custom flavors
• Fixed sorting logic when using custom fetching criteria
• Prevented inner fsspec logs from being flushed to the artifact store

Documentation

• Added LLM messaging and video resources to documentation
• Improved formatting for model deployers documentation
• Fixed GCP service connector docs
• Added SDK documentation links
• Enhanced README with LLM messaging

What's Changed

• Adding 0.74.0 to the migration tests by @bcdurak in #3351
• Fixing the release preparation workflow by @bcdurak in #3348
• Expand fastapi dependency range by @strickvl in #3340
• Document the programmatic API access options by @stefannica in #3352
• Fix some docs links by @schustmi in #3353
• [docs] Rename llms.txt file, add header and docs by @wjayesh in #3346
• Add llms.txt YouTube video to docs by @strickvl in #3354
• Fix model deployers docs formatting by @strickvl in #3356
• Get the latest artifact of a model by creation date instead of version name by @pierre-godard in #3343
• Improve source root docs/logs when running a pipeline by @schustmi in #3357
• Fix registration of components with custom flavors by @schustmi in #3363
• Fix GCP service connector docs by @stefannica in #3365
• Allow auth method to be customized during auto-configuration of service connectors by @stefannica in #3367
• Add some sdkdocs links by @htahir1 in #3358
• doc: fix link by @tanguyantoine in #3369
• Fix sorting when using custom fetching by @schustmi in #3366
• Add sagemaker env settings by @stefannica in #3368
• Update README with LLM messaging and llms.txt by @wjayesh in #3362
• CI Linting fix by @bcdurak in #3377
• Don't flush inner fsspec logs to the artifact store by @stefannica in #3373
• Bugfix for Sagemaker env variables by @bcdurak in #3380
• Correct isintance check for sagemaker env variables by @bcdurak in #3382

New Contributors

• @pierre-godard made their first contribution in #3343
• @tanguyantoine made their first contribution in #3369

Full Changelog: 0.74.0...0.75.0

0.74.0

The 0.74.0 release introduces several major features including SageMaker pipeline scheduling capabilities, Azure Container Registry (ACR) implicit authentication support, and Vertex AI persistent resource handling for step operators. Additionally, this release includes comprehensive improvements to timezone handling and significant enhancements to database performance.

Features

• API Tokens support in the dashboard for time-boxed API authentication
• SageMaker pipeline scheduling capabilities
• Azure Container Registry (ACR) and Storage Account implicit authentication
• Vertex AI persistent resource support for step operators
• Support for custom log formats
• Run metadata and tag indices for improved performance
• Core concepts video added to documentation

Improvements

• Comprehensive timezone consistency improvements across the platform
• Enhanced database query performance for pipelines, run templates, models, and artifacts
• Better handling of configured parameters during pipeline preparation
• Support for passing run configurations as dictionaries when triggering pipelines
• Enhanced sorting capabilities for columns with empty values in the dashboard
• Improved queries for pipelines, run templates, models, and artifacts
• Better filtering functionality for run metadata
• More efficient artifact filtering
• Various Helm chart improvements and reorganization
• Updated materializer support for newer PyTorch versions
• Improved code repository management and downloading
• Better handling of SecretStr values in store configurations

Fixes

• Kubernetes service connector issues resolved
• Fixed sorting for columns with potentially empty values
• Corrected timestamp utilization for better timezone consistency
• Resolved issues with vLLM pipeline config file usage
• Fixed code download functionality for custom flavor components
• Addressed various documentation and broken links
• Corrected MySQL database connection warnings
• Fixed issues with Vertex AI experiment tracker documentation

What's Changed

• Fix some docs by @htahir1 in #3302
• Replace deprecated datetime.utcnow() with datetime.now(timezone.utc) by @aiakide in #3265
• Adding the missing VertexAI experiment tracker docs by @bcdurak in #3308
• Create Sagemaker pipeline schedules if specified by @htahir1 in #3271
• Formatting by @schustmi in #3307
• Remove trailing slashes from zenml login URLs by @stefannica in #3312
• Fix Kubernetes service connector by @stefannica in #3313
• Add notes on missing features for on-prem ZenML Pro deployments by @stefannica in #3301
• Fix wrong warning log when directly connecting to MySQL DB by @schustmi in #3311
• Fix typo by @schustmi in #3316
• Minor fix for Sagemaker by @bcdurak in #3318
• Rework timestamp utilization for timezone consistency by @stefannica in #3314
• Add broken links checker by @htahir1 in #3305
• Schedule timezone fixes by @schustmi in #3315
• Misc code repository improvements by @schustmi in #3306
• Add core concepts video by @htahir1 in #3324
• Fix code download for custom flavor components by @schustmi in #3323
• Allow passing run configuration as dict when triggering pipelines by @schustmi in #3326
• Fix sorting by columns with potentially empty values by @schustmi in #3325
• Allow custom log formats by @schustmi in #3288
• Add vertex persistent resource to settings for step operator by @htahir1 in #3304
• Fix use of config file in vLLM pipelines by @wjayesh in #3322
• Fixing the CI with the new huggingface-hub version by @bcdurak in #3329
• Handling string values as SecretStrs in store configurations by @bcdurak in #3319
• More code repository improvements by @schustmi in #3327
• Fix materializer for new pytorch version by @schustmi in #3331
• Add some nicer docs by @htahir1 in #3328
• Add run metadata and tag indices by @schustmi in #3310
• Fix markdown link checker for external PRs by @schustmi in #3333
• feat: implement implicit authentication for ACR and Storage Account by @lukas-reining in #3274
• Add support for symlinks in GH download by @schustmi in #3332
• ZenML Helm chart improvements by @stefannica in #3320
• Move helm chart out of the source tree by @stefannica in #3338
• Add option to skip stack validation by @schustmi in #3337
• Improve queries for pipelines, run templates, models and artifacts by @schustmi in #3335
• Improve configured parameter detection when preparing pipeline by @schustmi in #3339
• Minor fix for the Artifact filter model by @bcdurak in #3334
• Allow (un)installing integrations with system-wide uv installations by @schustmi in #3342
• Fix filtering by run metadata by @schustmi in #3344

New Contributors

• @lukas-reining made their first contribution in #3274

Full Changelog: 0.73.0...0.74.0

0.73.0

The 0.73.0 release contains various changes and improvements, but most importantly it introduces the support to deploy and enroll un-managed ZenML Pro tenants in the ZenML Pro control plane (Helm deployment options, secure enrollment, CSRF tokens) and other features necessary for self-hosted, multi-domain ZenML Pro installations.

Other Features

• Vertex AI experiment tracker integration
• Experiment comparison tooling. See loom for a quick demo.
• Support for new Airflow KubernetesPodOperator import paths
• Updated Slack alerter implementation
• Independent memory resource configuration for migration pods in Helm charts

Improvements

• Added environment variable to allow non-ASCII characters in JSON dumps
• Removed gluon from MLflow log suppression list
• Enhanced resource reporting with automatic conversion
• Documentation updates for Kubeflow Pipelines and LLMs
• Various bugfixes for the ZenML dashboard

What's Changed

• On-prem Pro tenants: secure enrollment, CSRF tokens, and cross-domain authorization flow by @stefannica in #3264
• Fix the misc release actions by @schustmi in #3286
• Add 0.72.0 to the migration tests by @schustmi in #3285
• Fix links to Kubeflow Pipelines docs in kubeflow.md by @matemijolovic in #3289
• Add experiment comparison tool docs by @strickvl in #3287
• Fix broken links by @strickvl in #3291
• Add support for new Airflow KubernetesPodOperator import by @schustmi in #3295
• Updated Slack Alerter by @bcdurak in #3282
• Allow non-ASCII in JSON dump with env var by @Frank995 in #3257
• Remove gluon from mlflow log suppression list by @htahir1 in #3298
• Convert reportable resources if necessary by @schustmi in #3296
• Vertex AI Experiment Tracker Integration by @nkhusainov in #3260
• Document on-prem ZenML Pro deployments by @stefannica in #3294
• generate llms.txt for our docs by @wjayesh in #3273
• [helm] Independent setting of memory resources for migration pods by @wjayesh in #3281
• Prepare release 0.73.0 by @github-actions in #3300

New Contributors

• @matemijolovic made their first contribution in #3289
• @Frank995 made their first contribution in #3257
• @nkhusainov made their first contribution in #3260

Full Changelog: 0.72.0...0.73.0

0.72.0

The 0.72.0 release contains various bug fixes, performance improvements and improvements to our documentation.

What's Changed

• Fix typo in readme by @schustmi in #3247
• adding 0.71.0 to migration tests by @bcdurak in #3250
• Fix workload token expiration for cached steps/runs by @schustmi in #3243
• Implement wandb settings conversion for latest release by @schustmi in #3246
• Add CPU usage note to Modal docs by @strickvl in #3253
• Re-authenticate requests that failed authentication by @stefannica in #3256
• Add new toc by @htahir1 in #3255
• Add step run unique constraint by @schustmi in #3236
• Fix build reuse after stack updates by @schustmi in #3251
• Fix fetching run template using the client by @schustmi in #3258
• Improved deprecation messages for artifact configs and run metadata by @bcdurak in #3261
• Filtering and sorting by @bcdurak in #3230
• Fix hyperparam tuning docs by @stefannica in #3259
• Include user of latest run in pipeline response by @schustmi in #3262
• Create model versions server-side to avoid race conditions by @schustmi in #3254
• Fix request model validation by @schustmi in #3245
• Improve docs to encourage using secrets by @AlexejPenner in #3272
• Include service connector requirements in custom flavor registration by @schustmi in #3267
• Fix the onboarding state to account for zenml login by @stefannica in #3270
• Improve the efficiency of some SQL queries by @schustmi in #3263
• Fix broken link by @strickvl in #3276
• Bump NLP template by @schustmi in #3275
• Fixed and improved sorting by @bcdurak in #3266
• Add matplotlib visualization to ZenML dashboard by @htahir1 in #3278
• Fix azure integration by @schustmi in #3279

Full Changelog: 0.71.0...0.72.0

0.71.0

ZenML version 0.71.0 delivers a new Modal step operator integration as its core feature, enabling efficient cloud execution for ML pipelines with granular hardware configuration options. The release strengthens enterprise capabilities through improved token management and dashboard features while expanding artifact handling with dynamic naming and enhanced visualization support. Additionally, it includes various infrastructure improvements and bug fixes that enhance the platform's stability and usability, particularly around Docker connectivity, Kubernetes management, and service connector operations.

New Feature: Modal Step Operator Integration

ZenML now integrates with Modal, bringing lightning-fast cloud execution capabilities to your ML pipelines. This new step operator allows you to execute individual pipeline steps on Modal's specialized compute instances, offering notable speed, particularly for Docker image building and hardware provisioning. With simple configuration options, you can precisely specify hardware requirements like GPU type, CPU count, and memory for each step, making it ideal for resource-intensive ML workloads.

New Feature: AWS Image Builder

Don't want to worry about Docker locally? Now build images remotely in AWS. Docs: https://docs.zenml.io/stack-components/image-builders/aws

Other Highlights

• Workload API Token Management: Refactored token management for improved security with a generic API token dispenser.
• Dashboard Enhancements:
  • Introduced service account management capabilities.
  • Added API key creation and integration features.
• Dynamic Artifact Naming: Introduced capability to dynamically name artifacts.
• Visualization Enhancements: Made dictionaries and lists visualizable, added JSON visualization type.

Additional Features and Improvements

• Improved error messages for Docker daemon connectivity
• Enhanced SageMaker URL handling
• Simplified model version artifact linkage
• Added testing for pipeline templates
• Improved Kubernetes pod and label length management
• Allowed skipping type annotations for step inputs
• Enabled using feature service instances instead of just names

Bug Fixes

• Fixed issues with getting out of an inaccessible active stack
• Fixed race conditions in the service connector type registry
• Resolved migration test complications
• Corrected documentation links
• Fixed artifact store and artifact URI handling
• Addressed various scalability and compatibility issues

Documentation Updates

• Added documentation redirects
• Updated PyTorch documentation links
• Improved service connector documentation

What's Changed

• Refactored workload API token management for better security and implemented generic API token dispenser by @stefannica in #3154
• Add 0.70.0 to the migration tests by @avishniakov in #3190
• Adjustments to the PR template by @bcdurak in #3194
• [docs] Fix links in the how-to section of docs by @wjayesh in #3196
• Fixing sagemaker urls to take the settings into consideration by @bcdurak in #3195
• Add cached run into testing of migrations by @avishniakov in #3199
• Fix service connector type registry race conditions by @stefannica in #3202
• Refactor container resource configuration in Vertex Orchestrator test by @avishniakov in #3203
• [docs] Add missing redirects by @wjayesh in #3200
• Add links to uv new PyTorch documentation by @strickvl in #3204
• Fix broken docs link by @strickvl in #3208
• Bugfix for getting out of an inaccessible active stack with no permissions by @bcdurak in #3198
• Simplify model version artifact linkage by @schustmi in #3175
• Reenable macos testing by @avishniakov in #3205
• Various fixes and improvements by @stefannica in #3211
• Pass config path during zenml pipeline build by @schustmi in #3212
• Add test for running templates by @schustmi in #3192
• Fix service connector docstring by @schustmi in #3216
• Improve error message when docker daemon is not reachable by @schustmi in #3214
• Don't run migration for empty updates by @schustmi in #3210
• Remove --check from format script and fix naming by @safoinme in #3218
• More scalability improvements by @schustmi in #3206
• Use correct keyword for artifact store open by @schustmi in #3220
• Fix passing of some sagemaker settings by @schustmi in #3221
• Add hint when trying to connect with api key by @schustmi in #3222
• Allow passing None values as parameter for optional complex types by @schustmi in #3215
• Limit kubernetes pod and label length by @schustmi in #3217
• Updating the quickstart example to use the new log_metadata by @bcdurak in #3188
• Allow skipping type annotations for step inputs by @schustmi in #3223
• Modal Step Operator by @strickvl in #2948
• Add dynamic artifacts naming, documentation and tests by @avishniakov in #3201
• Run template CLI command and bugfix by @schustmi in #3225
• Make dicts/lists visualizable and add JSON as viz type by @wjayesh in #2882
• Instances of the FeatureServices are now used instead of only the names of the FeatureServices. by @aiakide in #3209
• Quickstart fixes by @schustmi in #3227
• Add missing docs by @schustmi in #3226
• Misc cleanup by @schustmi in #3229
• Fix input resolution for steps with dynamic artifact names by @schustmi in #3228
• Follow-up on the run_metadata changes by @bcdurak in #3193
• Fixed broken links by @htahir1 in #3232
• Fixed wandb login problem in Quickstart by @htahir1 in #3233
• Misc bugfixes by @schustmi in #3234
• Add additional way to fetch docker repo digest by @schustmi in #3231
• AWS Image Builder implementation by @stefannica in #2904
• Disable client-side caching for some orchestrators by @schustmi in #3235
• Fix artifact uris for artifacts with name placeholders by @schustmi in #3237
• Materializer test fix on Windows by @bcdurak in #3238
• Fix GET step run endpoint to return unhydrated response if requested by @schustmi in #3240
• Pipeline run API token fixes and improvements by @stefannica in #3242

Full Changelog: 0.70.0...0.71.0

0.70.0

The ZenML 0.70.0 release includes a significant number of database schema changes and migrations, which means upgrading to this version will require extra caution. As always, please make sure to make a copy of your production database before upgrading.

Key Changes

• Artifact Versioning Improvements: The handling of artifact versions has been improved, including the API improvements like the ability to batch artifact version requests to improve the execution times and more types for the step input/output artifacts, including multiple versions of the same artifact (e.g. model checkpoints), to improve the UX using ZenML UI or while working directly with the API.
• Scalability Enhancements: Various scalability improvements have been made, such as reducing unnecessary server requests and incrementing artifact versions server-side. These enhancements are expected to provide significant speed and scale improvements for ZenML users.
• Metadata management: Now, all the metadata-creating functions are gathered under one method called log_metadata. It is possible to call this method with different inputs to log run metadata for artifact versions, model versions, steps, and runs.
• The oneof filtering: This allows to filter entities using a new operator called oneof. You can use this with IDs (UUID type) or tags (or other string-typed attributes) like this PipelineRunFilter(tag='oneof:["cats", "dogs"]').
• Documentation Improvements: The ZenML documentation has been restructured and expanded, including the addition of new sections on finetuning and LLM/ML engineering resources.
• Bug Fixes: This release includes several bug fixes, including issues with in-process main module source loading, and more.

Caution: Make sure to back up your data before upgrading!

While this release brings many valuable improvements, the database schema changes and migrations pose a potential risk to users. It is strongly recommended that users:

• Test the upgrade on a non-production environment: Before upgrading a production system, test the upgrade process in a non-production environment to identify and address any issues.
• Back up your data: Ensure that you have a reliable backup of your ZenML data before attempting the upgrade.

What's Changed

• Optimizing the CI workflows by @bcdurak in #3145
• Adding 0.68.0 to the migration tests by @bcdurak in #3144
• Move step durations to body by @schustmi in #3046
• Docs on ZenML setup by @strickvl in #3100
• Remove wrongly set Model.was_created_in_this_run attribute by @schustmi in #3129
• Allow specifying run tags in pipeline configuration by @schustmi in #3130
• Fix materializer type compatibility check during loading by @schustmi in #3105
• [docs] Add icons to headers in docs by @wjayesh in #3149
• fix icons and remove redundant file by @wjayesh in #3150
• Merge 0.68.1 release into develop by @schustmi in #3153
• Allow filtering pipeline runs by stack component by @schustmi in #3142
• Allow artifact response as step input by @schustmi in #3134
• Filter component by user name by @schustmi in #3126
• [docs] Restructure how-to section to make it more readable by @wjayesh in #3147
• ZenML Pro web login implementation by @stefannica in #3141
• Scalability improvements: Reduce misc/hydration server requests by @schustmi in #3093
• Fix in-process main module source loading by @schustmi in #3119
• Catch assertion in GH library by @schustmi in #3160
• Enable cache precomputation for run templates by @schustmi in #3156
• Add LLM and ML engineering books to README by @htahir1 in #3159
• Add helper method to quickly create run template from pipeline by @schustmi in #3155
• Add CLI command to export stack requirements by @schustmi in #3158
• Scalability improvements: Increment artifact version server side by @schustmi in #3095
• Update OpenAI integration by @safoinme in #3163
• Remove deprecated torch version constraint by @safoinme in #3166
• vLLM model deployer by @dudeperf3ct in #3032
• Don't initialize client during flavor sync by @schustmi in #3168
• Cleanup materializer temporary directories after step execution by @schustmi in #3162
• Fix langchain in API docs by @avishniakov in #3171
• Finetuning guide by @strickvl in #3157
• Fix mypy issue vllm evidently by @safoinme in #3169
• Add artifact version batch request by @schustmi in #3164
• Add missing section links by @strickvl in #3172
• Fix uvloop mypy by @avishniakov in #3174
• Multiple output versions for a step outputs by @avishniakov in #3072
• Simplify Metadata handling by @AlexejPenner in #3096
• assign value to component_name in preset stack registration by @hirekk in #3178
• Updating the template versions with zenml login by @bcdurak in #3177
• Better input artifacts typing by @avishniakov in #3099
• Refactor environment setup and caching by @safoinme in #3077
• Fix spelling errors by @safoinme in #3181
• Prevent some race conditions by @schustmi in #3167
• Update stack deployments with latest features by @stefannica in #3183
• Terraform best practices by @htahir1 in #3131
• Fix sagemaker pipeline URLs by @stefannica in #3176
• Fix lightning orchestrator for multi-step pipelines by @wjayesh in #3170
• Port bugfixes from #2497 by @avishniakov in #3179
• Removing the enable_cache from the config files by @bcdurak in #3184
• Don't pass tags to step config by @schustmi in #3186
• New log_metadata function, new oneof filtering, additional run_metadata filtering by @bcdurak in #3182

New Contributors

• @hirekk made their first contribution in #3178

Full Changelog: 0.68.1...0.70.0

0.68.1

Bug fixes

Fixes an issue with some partially cached pipelines running on remote orchestrators.

What's Changed

• Remove unavailable upstream steps during cache precomputation by @schustmi in #3146

Full Changelog: 0.68.0...0.68.1

0.68.0

Highlights

• Stack Components on the Dashboard: We're bringing back stack components. With this release, you will get access to the list of your stack components on the ZenML dashboard. More functionality is going to follow in the next releases.
• Client-Side Caching: Implemented client-side computation for cached steps, significantly reducing time and costs associated with remote orchestrator spin-up.
• Streamlined Onboarding Process: Unified the starter and production setup into a single sequential flow, providing a more intuitive user experience.
• BentoML Integration: Updated to version 1.3.5 with enhanced containerization support.
• Artifact Management: Introduced register_artifact function enabling direct linking of existing data in the artifact store, particularly useful for tools like PyTorch-Lightning that manage their own checkpoints.
• Enhanced Error Handling: Added Error Boundary to visualization components for improved reliability and user experience.

Additional Features and Improvements

• Added multiple access points for deleting pipeline runs
• Improved pipeline detail view functionality
• Improved service account handling for Kaniko image builder

Breaking Changes and Deprecations

• Discontinued Python 3.8 support
• Removed legacy pipeline and step interface
• Removed legacy post execution workflow
• Removed legacy dashboard option
• Removed zenml stack up/down CLI commands
• Removed zenml deploy and zenml <stack-component> deploy
• Removed StepEnvironment class
• Removed the option to specify a specific model version for step output artifacts using the ArtifactConfig class
• Removed the option to use the ExternalArtifact class to load an artifact from a model version
• Removed Client.list_runs, replacing it with Client.list_pipeline_runs
• Removed ArtifactVersionResponse.read, replacing it with ArtifactVersionResponse.load

Documentation Updates

Added new guides for the following topics:

• Kubernetes per-pod configuration
• Factory generation of artifact names
• Common stacks best practices
• Azure 1-click dashboard deployment
• ZenML server upgrade best practices
• Custom Dataset classes and Materializers
• Comprehensive ZenML Pro documentation
• Image building optimization during pipeline runs
• Enhanced BentoML integration documentation

What's Changed

• Release 0.67.0 migration testing by @bcdurak in #3050
• Prevent too large requests by @avishniakov in #3048
• Fix Neptune linting after 1.12.0 release by @avishniakov in #3055
• Fix Lightning Orchestrator (remove -y from pip install) by @wjayesh in #3058
• Fix artifact pruning endpoint path by @schustmi in #3052
• Update python versioning in docs by @avishniakov in #3059
• Fix infinite loop while fetching artifact store in logs storage class by @avishniakov in #3061
• Make sync a setting for sagemaker/azureml orchestrator by @schustmi in #3062
• Remove some deprecated features by @schustmi in #2926
• Fix MySQL warning when filtering pipelines by latest run by @schustmi in #3051
• Remove more deprecated stuff by @schustmi in #3063
• Remove log versions from versioned buckets in S3 by @avishniakov in #3060
• add docs on k8s per pod settings by @wjayesh in #3066
• Remove Python 3.8 support by @strickvl in #3034
• register_artifact function by @avishniakov in #3053
• Fix bad link in docs by @avishniakov in #3069
• Fix model linkage for the lazy loading scenarios by @avishniakov in #3054
• Updating template versions after the Python 3.8 changes by @bcdurak in #3070
• Add UUID materializer by @htahir1 in #3073
• Fix pipeline and model URLs for ZenML Pro on-prem deployments by @stefannica in #3083
• Update bentoml integration to 1.3.5 and add containerization by @wjayesh in #3045
• Fix mlflow linting by @schustmi in #3085
• Add docs for factory generation of artifact names by @strickvl in #3084
• Remove unnecessary metadata fields in UUID materializer test by @htahir1 in #3088
• Client-side computation of cached steps by @schustmi in #3068
• Fix Kaniko image builder service account passing by @schustmi in #3081
• Bugfix in GitLab Code Repository integration by @4gt-104 in #3076
• Add docs on common stacks best practices by @strickvl in #3092
• [docs] Update stacks page and add azure 1-click from dashboard docs by @wjayesh in #3082
• Local development how-to section by @strickvl in #3090
• [docs] best practices for upgrading zenml server by @wjayesh in #3087
• Fix S3 ArtifactStore auth issue by @avishniakov in #3086
• Reduce migration testing runtime by @avishniakov in #3078
• [docs] Dedicated docs on how to skip building an image on pipeline run by @wjayesh in #3079
• Fix filtering by tag for pipeline runs by @schustmi in #3097
• Remove deprecated features: zenml deploy and zenml <stack-component> deploy by @stefannica in #3089
• Do not tag model via Model class on creation by @avishniakov in #3098
• Sagemaker add pipeline tags by @htahir1 in #3080
• [docs] Add custom Dataset classes and Materializers in ZenML by @htahir1 in #3091
• Delete Scarf related scripts and workflow files by @htahir1 in #3103
• Add more detailed docs for ZenML Pro by @wjayesh in #3065
• Add missing code hash filter in client method by @schustmi in #3094
• Remove lineage graph and legacy dashboard support by @schustmi in #3064
• Add unittest to cover gitlab CR regex. by @4gt-104 in #3102
• Automating the release process using Github workflows by @bcdurak in #3101
• Bugfix for release automation by @bcdurak in #3107
• Bugfix for new version in the release automation by @bcdurak in #3108
• using the right parent image name by @bcdurak in #3109
• Making the new release automation scripts executable by @bcdurak in #3110
• Fixing the env variables for the release automation by @bcdurak in #3111
• Adding the right Github configuration before using the gh CLI to fetch the version by @bcdurak in #3112
• Fixing the outputs of the first step in the release automation by @bcdurak in #3113
• Handling github auth and release notes for release automation by @bcdurak in #3114
• Fixing the cloudbuild call for release automation by @bcdurak in #3116
• Fixing the update tenant call in the script by @bcdurak in #3118
• Release automation with the new redeploy logic by @bcdurak in #3120
• Fixing the automation triggers for other branches by @bcdurak in #3125
• Update link for llm-complete-guide repository.- Updated link to poi… by @htahir1 in #3128
• Fixing the migration testing for the release branches by @bcdurak in #3127
• Update pipeline deletion docs by @strickvl in #3123
• Disabling the cache for the quickstart tests by @bcdurak in #3133
• Update Argilla integration for v2.x SDK by @sdiazlor in #2915
• Using pip instead of gh CLI in the migration tests by @bcdurak in #3136
• Adapting tags to work with older versions of Sagemaker by @bcdurak in #3135
• Manual trigger for the release_finalize workflow by @bcdurak in #3137
• Fixing the prepare trigger for the release automation by @bcdurak in #3138

New Contributors

• @4gt-104 made their first contribution in #3076
• @sdiazlor made their first contribution in #2915

Full Changelog: 0.67.0...0.68.0