Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Update various library versions to meet the minimum required CMake version (>= 3.5) for compatibility with CMake 3.30.4. #858

Merged
merged 18 commits into from
Apr 3, 2025

Conversation

gigony
Copy link
Contributor

@gigony gigony commented Mar 31, 2025

This patch resolves the CMake errors caused by the minimum required CMake version (>= 3.5) when using CMake 3.30.4.

This patch upgrades fmt (7.0.1 -> 10.1.1), googletest (1.10.0 -> 1.16.0), cli11 (1.9.1 -> 2.5.0), json (3.9.1 -> 3.11.3), libjpeg-turbo (patch CMakeLists.txt), libopenjpeg(2.5.0 -> 2.5.3, patch CMakeLists.txt), libtiff(4.1.0 -> 4.5.0 -> 4.1.0, patch CMakeLists.txt), pugixml(1.11.1 -> 1.15), pybind11_json (0.2.9 -> 0.2.15), libcuckoo(patch CMakeLists.txt)

Cloning into 'deps-fmt-src'...
HEAD is now at f19b1a5 Update version
CMake Error at build-release/_deps/deps-fmt-src/CMakeLists.txt:1 (cmake_minimum_required):
  Compatibility with CMake < 3.5 has been removed from CMake.

  Update the VERSION argument <min> value.  Or, use the <min>...<max> syntax
  to tell CMake that the project requires at least <min> but has been updated
  to work with policies introduced by <max> or earlier.

  Or, add -DCMAKE_POLICY_VERSION_MINIMUM=3.5 to try configuring anyway.


-- Configuring incomplete, errors occurred!
Cloning into 'deps-googletest-src'...
HEAD is now at 703bd9c Googletest export
CMake Error at build-release/_deps/deps-googletest-src/CMakeLists.txt:4 (cmake_minimum_required):
  Compatibility with CMake < 3.5 has been removed from CMake.

  Update the VERSION argument <min> value.  Or, use the <min>...<max> syntax
  to tell CMake that the project requires at least <min> but has been updated
  to work with policies introduced by <max> or earlier.

  Or, add -DCMAKE_POLICY_VERSION_MINIMUM=3.5 to try configuring anyway.
Cloning into '/__w/cucim/cucim/build-release/_deps/deps-cli11-src/extern/googletest'...
Submodule path 'extern/googletest': checked out '859bfe8981d6724c4ea06e73d29accd8588f3230'
CMake Error at build-release/_deps/deps-cli11-src/CMakeLists.txt:1 (cmake_minimum_required):
  Compatibility with CMake < 3.5 has been removed from CMake.

  Update the VERSION argument <min> value.  Or, use the <min>...<max> syntax
  to tell CMake that the project requires at least <min> but has been updated
  to work with policies introduced by <max> or earlier.

  Or, add -DCMAKE_POLICY_VERSION_MINIMUM=3.5 to try configuring anyway.


-- Configuring incomplete, errors occurred!
CMake Error at build-release/_deps/deps-json-src/CMakeLists.txt:1 (cmake_minimum_required):
  Compatibility with CMake < 3.5 has been removed from CMake.

  Update the VERSION argument <min> value.  Or, use the <min>...<max> syntax
  to tell CMake that the project requires at least <min> but has been updated
  to work with policies introduced by <max> or earlier.

  Or, add -DCMAKE_POLICY_VERSION_MINIMUM=3.5 to try configuring anyway.

Note that libtiff should be kept at v4.1.0 to avoid build and test issues.

It’s possible that this is caused by the following PR and the CMake version upgrade:

Signed-off-by: Gigon Bae <gbae@nvidia.com>
@gigony gigony added non-breaking Introduces a non-breaking change maintenance labels Mar 31, 2025
@gigony gigony requested a review from a team as a code owner March 31, 2025 22:19
@gigony gigony added the improvement Improves an existing functionality label Mar 31, 2025
Signed-off-by: Gigon Bae <gbae@nvidia.com>
@gigony gigony changed the title Update fmt library version to 10.1.1 Update fmt/googletest library version Mar 31, 2025
Signed-off-by: Gigon Bae <gbae@nvidia.com>
@gigony gigony changed the title Update fmt/googletest library version Update fmt/googletest/cli11 library version Mar 31, 2025
@gigony gigony changed the title Update fmt/googletest/cli11 library version Update fmt/googletest/cli11/json library version Mar 31, 2025
gigony added 4 commits March 31, 2025 16:09
Signed-off-by: Gigon Bae <gbae@nvidia.com>
Updated error reporting in CUDA and NVJPEG functions to cast status codes to integers for clearer output. This change enhances the readability of error messages by ensuring that the status codes are displayed as integers rather than their default types.
Without this changes, it causes error, demanding the template method for
the custom types.

Signed-off-by: Gigon Bae <gbae@nvidia.com>
This change ensures compatibility with newer features and improvements in CMake, enhancing the build process for the libcuckoo library.

Signed-off-by: Gigon Bae <gbae@nvidia.com>
Do not use catch_discover_tests() since it causes a test to be run at build time
and somehow it causes a deadlock/segfault during the build.

Signed-off-by: Gigon Bae <gbae@nvidia.com>
@gigony gigony requested a review from a team as a code owner April 1, 2025 00:07
@gigony gigony changed the title Update fmt/googletest/cli11/json library version Update various library versions to meet the minimum required CMake version (>= 3.5) for compatibility with CMake 3.30.4. Apr 1, 2025
gigony added 6 commits March 31, 2025 17:25
Signed-off-by: Gigon Bae <gbae@nvidia.com>
Signed-off-by: Gigon Bae <gbae@nvidia.com>
Signed-off-by: Gigon Bae <gbae@nvidia.com>
Signed-off-by: Gigon Bae <gbae@nvidia.com>
Refactor TIFF_SSIZE_T definition and update data types in IFD class
- Added a check to ensure TIFF_SSIZE_T is only defined if not already defined, changing its type to int64_t.
- Updated the data types of td_stripoffset_p and td_stripbytecount_p from uint64 to uint64_t for consistency in the IFD class.

Signed-off-by: Gigon Bae <gbae@nvidia.com>
@gigony gigony force-pushed the gbae/update_fmt branch 3 times, most recently from 94bc426 to 905267d Compare April 1, 2025 00:38
gigony added 2 commits March 31, 2025 17:54
- Introduced a patch command in libtiff.cmake to apply custom modifications.
- Updated the minimum required CMake version to 3.30.0 in the CMakeLists.txt file.

Signed-off-by: Gigon Bae <gbae@nvidia.com>
- Introduced a patch command in libopenjpeg.cmake to apply custom modifications.
- Updated the minimum required CMake version to 3.30.0 in the CMakeLists.txt file.

Signed-off-by: Gigon Bae <gbae@nvidia.com>
@gigony gigony force-pushed the gbae/update_fmt branch from 905267d to 0bc5648 Compare April 1, 2025 03:48
- Change the way environment variables are set for SIMD flags by using static character arrays to improve memory management.
- Update row pointer assignments to cast to JSAMPROW for better type safety.

Signed-off-by: Gigon Bae <gbae@nvidia.com>
Signed-off-by: Gigon Bae <gbae@nvidia.com>
@gigony gigony assigned gigony and unassigned grlee77 Apr 1, 2025
@gigony gigony requested review from grlee77 and bdice April 1, 2025 05:42
Copy link
Contributor

@grlee77 grlee77 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I had some question about whether all the patches are necessary. I see based on the chain of failed builds that it was not working without them, but wondered if there might be a more minimal fix. No problem with merging it as-is if you don't see another option, but was just wondering about how much extra work it might be to keep updating patches going forward.

Comment on lines 10 to 17
+# [cuCIM patch] Set minimum CMake version to 3.30.0
+cmake_minimum_required(VERSION 3.30.0)

-# Default policy is from 2.8.9
-cmake_policy(VERSION 2.8.9)
+# [cuCIM patch] Set default policy to 3.30.0
+# Default policy is from 3.30.0
+cmake_policy(VERSION 3.30.0)
Copy link
Contributor

@grlee77 grlee77 Apr 1, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It seems since CMake 3.12 it has been possible to specify a range of versions for cmake_minimum_required and cmake_policy (docs page). Not sure if that is a better approach, but what I didn't understand is:

1.) Why does having a lower minimum for cmake_minimum_required cause the build problems? 3.30 is greater than 2.8.9 so it seems like it wouldn't cause a conflict.

2.) For set_policy, another option mentioned on that page is environment variable CMAKE_POLICY_VERSION_MINIMUM could override the policy.

I was wondering if we set that environment variable during the build if some of the patches could be removed?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Oh, I see in that same docs page that it states:

Changed in version 3.27: Compatibility with versions of CMake older than 3.5 is deprecated. Calls to cmake_minimum_required(VERSION) or cmake_policy(VERSION) that do not specify at least 3.5 as their policy version (optionally via ...) will produce a deprecation warning in CMake 3.27 and above.

So it seems ideally upstream projects would add some maximum version like

cmake_minimum_required(2.8.9...3.30.0)

but unless that is implemented upstream we will still need some patch.

Maybe just check whether setting CMAKE_POLICY_VERSION_MINIMUM allows removing some of the patches?

Copy link
Contributor Author

@gigony gigony Apr 1, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks @grlee77 for the investigation and suggestion!

Maybe just check whether setting CMAKE_POLICY_VERSION_MINIMUM allows removing some of the patches?

Indeed, setting it could eliminate the need for some of the patches.
According to the manual (https://cmake.org/cmake/help/latest/variable/CMAKE_POLICY_VERSION_MINIMUM.html),

This variable should not be set by a project in CMake code as a way to set its own policy version. Use cmake_minimum_required(VERSION) and/or cmake_policy(VERSION) for that. This variable is meant to externally set policies for which a project has not itself been updated:

This variable needs to be set by an external command, and something like the following could bypass the error:

--- a/run
+++ b/run
@@ -281,7 +281,8 @@ build_local_libcucim_() {
         -DCMAKE_EXPORT_COMPILE_COMMANDS:BOOL=TRUE \
         -DCMAKE_PREFIX_PATH=${prefix} \
         -DCMAKE_BUILD_TYPE=${build_type_str} \
-        -DCMAKE_INSTALL_PREFIX=${source_folder}/install
+        -DCMAKE_INSTALL_PREFIX=${source_folder}/install \
+        -DCMAKE_POLICY_VERSION_MINIMUM=3.30.0
     ${CMAKE_CMD} --build ${build_folder} --config ${build_type_str} --target cucim -- -j $(nproc)
     ${CMAKE_CMD} --build ${build_folder} --config ${build_type_str} --target install -- -j $(nproc)

Projects may set this variable before a call to add_subdirectory() that adds a third-party project in order to set its policy version without modifying third-party code.

I couldn't find a proper way to set the variable for FetchContent_MakeAvailable() method (in such as 'cpp/cmake/deps/libcuckoo.cmake').

I am afraid that setting CMAKE_POLICY_VERSION_MINIMUM in the build command could make the following change meaningless:

Unless the API is broken or the update requires significant code changes, I believe updating libraries is beneficial from a security standpoint and for future work. I think resolving the minimum requirement issue by upgrading the versions is a reasonable approach, and I'd prefer to keep the current PR as is.

@gigony
Copy link
Contributor Author

gigony commented Apr 1, 2025

Hi @bdice, could you please help review (and possibly merge) this PR?

Copy link
Contributor

@bdice bdice left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I am okay with these changes. I asked @KyleFromNVIDIA to take a second look, since there are a lot of CMake updates, and there may be a better way to structure the patches. If he's happy, let's merge.

@@ -26,6 +26,7 @@ if (NOT TARGET deps::libjpeg-turbo)
GIT_REPOSITORY https://github.com/libjpeg-turbo/libjpeg-turbo.git
GIT_TAG 2.0.6
GIT_SHALLOW TRUE
PATCH_COMMAND ${GIT_EXECUTABLE} apply "${CMAKE_CURRENT_LIST_DIR}/libjpeg-turbo.patch"
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Shouldn't we be generating this with rapids_cpm_generate_patch_command()?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

cuCIM doesn't really use rapids-cmake today. I think that is the right idea, but there's a lot of effort needed to get cuCIM onto rapids-cmake.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks @KyleFromNVIDIA and @bdice!
Yes, currently cuCIM doesn't follow the standard RAPIDS build setup, but I’m planning to transition to a rapids-cmake and conda-package-based build setup in the near future.

@@ -18,7 +18,8 @@ if (NOT TARGET deps::libopenjpeg)
FetchContent_Declare(
deps-libopenjpeg
GIT_REPOSITORY https://github.com/uclouvain/openjpeg.git
GIT_TAG v2.5.0
GIT_TAG v2.5.3
PATCH_COMMAND ${GIT_EXECUTABLE} apply "${CMAKE_CURRENT_LIST_DIR}/libopenjpeg.patch"
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Shouldn't we be generating this with rapids_cpm_generate_patch_command()?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@@ -26,6 +26,7 @@ if (NOT TARGET deps::libtiff)
GIT_REPOSITORY https://gitlab.com/libtiff/libtiff.git
GIT_TAG v4.1.0
GIT_SHALLOW TRUE
PATCH_COMMAND ${GIT_EXECUTABLE} apply "${CMAKE_CURRENT_LIST_DIR}/libtiff.patch"
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Shouldn't we be generating this with rapids_cpm_generate_patch_command()?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Comment on lines 15 to 20
+# [cuCIM patch] Set default policy to 3.30.0
+# Default policy is from 3.30.0
+cmake_policy(VERSION 3.30.0)
# Set MacOSX @rpath usage globally.
if (POLICY CMP0020)
cmake_policy(SET CMP0020 NEW)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

By setting cmake_minimum_required() to 3.30.0, all of this is redundant. It sets the policy level to 3.30.0, which includes setting CMP0020 to NEW.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank

By setting cmake_minimum_required() to 3.30.0, all of this is redundant. It sets the policy level to 3.30.0, which includes setting CMP0020 to NEW.

Thanks @KyleFromNVIDIA for the information and feedback!

Let me also patch those redundant lines.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just updated. Thank you!

+
+# [cuCIM patch] Set default policy to 3.30.0
+# Default policy is from 3.30.0
+cmake_policy(VERSION 3.30.0)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

cmake_policy() is also redundant, because cmake_minimum_required() already set it above.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks @KyleFromNVIDIA !
Updated.

Signed-off-by: Gigon Bae <gbae@nvidia.com>
@gigony gigony force-pushed the gbae/update_fmt branch from 0bd4082 to 8196f03 Compare April 2, 2025 21:31
@gigony gigony requested a review from KyleFromNVIDIA April 2, 2025 23:18
@grlee77
Copy link
Contributor

grlee77 commented Apr 3, 2025

Thanks @gigony for the feature and @bdice and @KyleFromNVIDIA for reviewing

@grlee77
Copy link
Contributor

grlee77 commented Apr 3, 2025

/merge

@rapids-bot rapids-bot bot merged commit 4f60391 into rapidsai:branch-25.04 Apr 3, 2025
48 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
improvement Improves an existing functionality maintenance non-breaking Introduces a non-breaking change
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants