-
Notifications
You must be signed in to change notification settings - Fork 66
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Update various library versions to meet the minimum required CMake version (>= 3.5) for compatibility with CMake 3.30.4. #858
Conversation
Signed-off-by: Gigon Bae <gbae@nvidia.com>
Signed-off-by: Gigon Bae <gbae@nvidia.com>
Signed-off-by: Gigon Bae <gbae@nvidia.com>
Signed-off-by: Gigon Bae <gbae@nvidia.com>
Updated error reporting in CUDA and NVJPEG functions to cast status codes to integers for clearer output. This change enhances the readability of error messages by ensuring that the status codes are displayed as integers rather than their default types. Without this changes, it causes error, demanding the template method for the custom types. Signed-off-by: Gigon Bae <gbae@nvidia.com>
This change ensures compatibility with newer features and improvements in CMake, enhancing the build process for the libcuckoo library. Signed-off-by: Gigon Bae <gbae@nvidia.com>
Do not use catch_discover_tests() since it causes a test to be run at build time and somehow it causes a deadlock/segfault during the build. Signed-off-by: Gigon Bae <gbae@nvidia.com>
Signed-off-by: Gigon Bae <gbae@nvidia.com>
Signed-off-by: Gigon Bae <gbae@nvidia.com>
Signed-off-by: Gigon Bae <gbae@nvidia.com>
Signed-off-by: Gigon Bae <gbae@nvidia.com>
Signed-off-by: Gigon Bae <gbae@nvidia.com>
Refactor TIFF_SSIZE_T definition and update data types in IFD class - Added a check to ensure TIFF_SSIZE_T is only defined if not already defined, changing its type to int64_t. - Updated the data types of td_stripoffset_p and td_stripbytecount_p from uint64 to uint64_t for consistency in the IFD class. Signed-off-by: Gigon Bae <gbae@nvidia.com>
94bc426
to
905267d
Compare
- Introduced a patch command in libtiff.cmake to apply custom modifications. - Updated the minimum required CMake version to 3.30.0 in the CMakeLists.txt file. Signed-off-by: Gigon Bae <gbae@nvidia.com>
- Introduced a patch command in libopenjpeg.cmake to apply custom modifications. - Updated the minimum required CMake version to 3.30.0 in the CMakeLists.txt file. Signed-off-by: Gigon Bae <gbae@nvidia.com>
- Change the way environment variables are set for SIMD flags by using static character arrays to improve memory management. - Update row pointer assignments to cast to JSAMPROW for better type safety. Signed-off-by: Gigon Bae <gbae@nvidia.com>
Signed-off-by: Gigon Bae <gbae@nvidia.com>
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I had some question about whether all the patches are necessary. I see based on the chain of failed builds that it was not working without them, but wondered if there might be a more minimal fix. No problem with merging it as-is if you don't see another option, but was just wondering about how much extra work it might be to keep updating patches going forward.
+# [cuCIM patch] Set minimum CMake version to 3.30.0 | ||
+cmake_minimum_required(VERSION 3.30.0) | ||
|
||
-# Default policy is from 2.8.9 | ||
-cmake_policy(VERSION 2.8.9) | ||
+# [cuCIM patch] Set default policy to 3.30.0 | ||
+# Default policy is from 3.30.0 | ||
+cmake_policy(VERSION 3.30.0) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
It seems since CMake 3.12 it has been possible to specify a range of versions for cmake_minimum_required
and cmake_policy
(docs page). Not sure if that is a better approach, but what I didn't understand is:
1.) Why does having a lower minimum for cmake_minimum_required
cause the build problems? 3.30 is greater than 2.8.9 so it seems like it wouldn't cause a conflict.
2.) For set_policy
, another option mentioned on that page is environment variable CMAKE_POLICY_VERSION_MINIMUM
could override the policy.
I was wondering if we set that environment variable during the build if some of the patches could be removed?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Oh, I see in that same docs page that it states:
Changed in version 3.27: Compatibility with versions of CMake older than 3.5 is deprecated. Calls to cmake_minimum_required(VERSION) or cmake_policy(VERSION) that do not specify at least 3.5 as their policy version (optionally via ...) will produce a deprecation warning in CMake 3.27 and above.
So it seems ideally upstream projects would add some maximum version like
cmake_minimum_required(2.8.9...3.30.0)
but unless that is implemented upstream we will still need some patch.
Maybe just check whether setting CMAKE_POLICY_VERSION_MINIMUM
allows removing some of the patches?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thanks @grlee77 for the investigation and suggestion!
Maybe just check whether setting CMAKE_POLICY_VERSION_MINIMUM allows removing some of the patches?
Indeed, setting it could eliminate the need for some of the patches.
According to the manual (https://cmake.org/cmake/help/latest/variable/CMAKE_POLICY_VERSION_MINIMUM.html),
This variable should not be set by a project in CMake code as a way to set its own policy version. Use cmake_minimum_required(VERSION) and/or cmake_policy(VERSION) for that. This variable is meant to externally set policies for which a project has not itself been updated:
This variable needs to be set by an external command, and something like the following could bypass the error:
--- a/run
+++ b/run
@@ -281,7 +281,8 @@ build_local_libcucim_() {
-DCMAKE_EXPORT_COMPILE_COMMANDS:BOOL=TRUE \
-DCMAKE_PREFIX_PATH=${prefix} \
-DCMAKE_BUILD_TYPE=${build_type_str} \
- -DCMAKE_INSTALL_PREFIX=${source_folder}/install
+ -DCMAKE_INSTALL_PREFIX=${source_folder}/install \
+ -DCMAKE_POLICY_VERSION_MINIMUM=3.30.0
${CMAKE_CMD} --build ${build_folder} --config ${build_type_str} --target cucim -- -j $(nproc)
${CMAKE_CMD} --build ${build_folder} --config ${build_type_str} --target install -- -j $(nproc)
Projects may set this variable before a call to add_subdirectory() that adds a third-party project in order to set its policy version without modifying third-party code.
I couldn't find a proper way to set the variable for FetchContent_MakeAvailable()
method (in such as 'cpp/cmake/deps/libcuckoo.cmake').
I am afraid that setting CMAKE_POLICY_VERSION_MINIMUM
in the build command could make the following change meaningless:
Unless the API is broken or the update requires significant code changes, I believe updating libraries is beneficial from a security standpoint and for future work. I think resolving the minimum requirement issue by upgrading the versions is a reasonable approach, and I'd prefer to keep the current PR as is.
Hi @bdice, could you please help review (and possibly merge) this PR? |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I am okay with these changes. I asked @KyleFromNVIDIA to take a second look, since there are a lot of CMake updates, and there may be a better way to structure the patches. If he's happy, let's merge.
@@ -26,6 +26,7 @@ if (NOT TARGET deps::libjpeg-turbo) | |||
GIT_REPOSITORY https://github.com/libjpeg-turbo/libjpeg-turbo.git | |||
GIT_TAG 2.0.6 | |||
GIT_SHALLOW TRUE | |||
PATCH_COMMAND ${GIT_EXECUTABLE} apply "${CMAKE_CURRENT_LIST_DIR}/libjpeg-turbo.patch" |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Shouldn't we be generating this with rapids_cpm_generate_patch_command()
?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
cuCIM doesn't really use rapids-cmake today. I think that is the right idea, but there's a lot of effort needed to get cuCIM onto rapids-cmake.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thanks @KyleFromNVIDIA and @bdice!
Yes, currently cuCIM doesn't follow the standard RAPIDS build setup, but I’m planning to transition to a rapids-cmake and conda-package-based build setup in the near future.
@@ -18,7 +18,8 @@ if (NOT TARGET deps::libopenjpeg) | |||
FetchContent_Declare( | |||
deps-libopenjpeg | |||
GIT_REPOSITORY https://github.com/uclouvain/openjpeg.git | |||
GIT_TAG v2.5.0 | |||
GIT_TAG v2.5.3 | |||
PATCH_COMMAND ${GIT_EXECUTABLE} apply "${CMAKE_CURRENT_LIST_DIR}/libopenjpeg.patch" |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Shouldn't we be generating this with rapids_cpm_generate_patch_command()
?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thanks @KyleFromNVIDIA
Created the follow-up issue:
@@ -26,6 +26,7 @@ if (NOT TARGET deps::libtiff) | |||
GIT_REPOSITORY https://gitlab.com/libtiff/libtiff.git | |||
GIT_TAG v4.1.0 | |||
GIT_SHALLOW TRUE | |||
PATCH_COMMAND ${GIT_EXECUTABLE} apply "${CMAKE_CURRENT_LIST_DIR}/libtiff.patch" |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Shouldn't we be generating this with rapids_cpm_generate_patch_command()
?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thanks @KyleFromNVIDIA
Created the follow-up issue:
+# [cuCIM patch] Set default policy to 3.30.0 | ||
+# Default policy is from 3.30.0 | ||
+cmake_policy(VERSION 3.30.0) | ||
# Set MacOSX @rpath usage globally. | ||
if (POLICY CMP0020) | ||
cmake_policy(SET CMP0020 NEW) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
By setting cmake_minimum_required()
to 3.30.0
, all of this is redundant. It sets the policy level to 3.30.0
, which includes setting CMP0020
to NEW
.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thank
By setting
cmake_minimum_required()
to3.30.0
, all of this is redundant. It sets the policy level to3.30.0
, which includes settingCMP0020
toNEW
.
Thanks @KyleFromNVIDIA for the information and feedback!
Let me also patch those redundant lines.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Just updated. Thank you!
+ | ||
+# [cuCIM patch] Set default policy to 3.30.0 | ||
+# Default policy is from 3.30.0 | ||
+cmake_policy(VERSION 3.30.0) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
cmake_policy()
is also redundant, because cmake_minimum_required()
already set it above.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thanks @KyleFromNVIDIA !
Updated.
Signed-off-by: Gigon Bae <gbae@nvidia.com>
Thanks @gigony for the feature and @bdice and @KyleFromNVIDIA for reviewing |
/merge |
This patch resolves the CMake errors caused by the minimum required CMake version (>= 3.5) when using CMake 3.30.4.
This patch upgrades fmt (7.0.1 -> 10.1.1), googletest (1.10.0 -> 1.16.0), cli11 (1.9.1 -> 2.5.0), json (3.9.1 -> 3.11.3), libjpeg-turbo (patch CMakeLists.txt), libopenjpeg(2.5.0 -> 2.5.3, patch CMakeLists.txt), libtiff(4.1.0 -> 4.5.0 -> 4.1.0, patch CMakeLists.txt), pugixml(1.11.1 -> 1.15), pybind11_json (0.2.9 -> 0.2.15), libcuckoo(patch CMakeLists.txt)
Note that libtiff should be kept at v4.1.0 to avoid build and test issues.
It’s possible that this is caused by the following PR and the CMake version upgrade: