v2025.3.12.15.01

DataFed-v2025.3.12.15.01 Pre-release

Pre-release

MAJOR Breaking changes

MINOR Feature

[987] - This implements serialized GitLab CI pipelines
[918] - Add admin script for making a user an admin
[1009] - Add collections to database for holding Globus tokens.
[1002] - Add backoff for task worker when database fails to prevent log overflow.
[1109] - Add support for Globus token association to Globus collection and user.
[1215] - Add support for fetching Globus tokens associated with Globus collection and user.
[1214] - Add support for mapped collections on transfer tasks

PATCH Bug fixes/Technical Debt/Documentation

[984] - Fixes {server_default} from showing up in path.
[990] - Will stop running containers so that we can prune them.
[988] - Removed non-working subscribe function.
[958] - Addresses issues raised by static code analyzer.
[962] - Adds script that will check that docker image is in registry
[1015] - Uses abs path to ci pipeline script for gcs build jobs
[1013] - add set +e and set -e around is active check for arango service
[1011] - Add python script and ci job to clean up GCS node keys.
[1023] - Address unbound variables in harbor ci script.
[1027] - Fix clear_db.sh script by replacing console.log with print
[1012] - Allow customized base build image for Docker dependencies and runtime Dockerfiles
[986] - Design improvement to upload and download transfer box.
[985] - Handles longer than needed timeouts on ui pages.
[1053] - Set CMake to enable foxx tests when built in the CI.
[1086] - Address XML-RPC bug from deprecated client
[1149] - Docker container GCS Collection Mount Bug Fix
[1168] - Add authz unit testing to the CI
[1200] - Add JavaScript linter (eslint) and (prettier) formatter for JavaScript
[1180] - Refactor of authz foxx module, split into objects and added unit tests
[1223] - Fix deprecated method usage for Protobuf in Python client library
[1257] - Bug in authz library parsing of authz.conf file, globus paths incorrectly sanitized when using just '/'
[1255] - Fixes bug, in libauthz with log_path var not being read in from config file.
[1268] - Bug fixes unbound variable in foxx entrypoint file.
[1269] - Update Arangodb to 3.12.4
[1288] - Bug Jupyter Notebook in documentation were fixed.
[1321] - Refactor, allow core config threads via env vars

DataFed-v2025.2.16.21.15

PATCH Bug fixes/Technical Debt/Documentation

#1322 Fix configuration setting for allowing a custom number of threads.

DataFed-v2025.2.12.14.03

Release

MAJOR Breaking changes

MINOR Feature

#1240 [DLT-1110] Mapped Collection Endpoint Browse (1/4)
#1231 Revert "[DLT-1110] Mapped Collection Endpoint Browse (1/4)"
#1228 Feature DLT 1120 retrieve transfer token
#1224 [DAPS, Foxx] - 1180 refactor Part 2 authz
#1218 1216 daps feature add improved ci scanning
#1217 Feature DLT 1127 relay new token data
#1210 1209 feature add jsdoc linter
#1196 Add support for authz lookup
#1194 Feature DLT 1109 foxx new token data
#1192 Add DataFed dependency paths to curl so SSL will be found
#1160 1106 updated proto messages for setting token
#1141 DLT 1119 Provide method for Globus Authorize URL

PATCH Bug fixes/Technical Debt/Documentation

#1298 Fix bug in CI scripts associated with repo and gcs image build
#1239 Applied fix to deprecation warning
#1289 Add github action for linting jupyter
#1286 Update the schema documentation to improve the usability.
#1291 Update python notebooks
#1270 Add env var
#1271 Update Arangodb packaged version to using 3.12.4
#1274 Fix confusion in key uses in end to end tests
#1260 1255 feature mock core server initial integration testing authz
#1262 Address JSDoc complaints
#1258 Fix edge case in GridFTP Authz where '/' is used
#1251 Bug daps 1243 datafed web logging
#1238 Throw error if OpenStack error code returned from API
#1234 Fix bug on task_router.js abort function
#1227 Update nlhoman json version
#1221 1219 daps bug gitlab ci gcs log
#1212 1211 fix jsdoc errors
#1206 1205 refactor formatting across javascript
#1201 1200 feature GitHub action javascript linter formatter
#1197 Refactor foxx unit tests so they will run correctly
#1203 Address eslint errors
#1188 Remove --dump flag
#1187 1184 bug add set e foxx install
#1186 Changes to allow for and auto generate debug log level
#1181 1180 refactor foxx authz
#1175 Add -e back to core/database/test/scripts
#1169 1168 feature add authz unit tests to ci
#1167 Add missing comment on env variable
#1161 1159 refactor authz cpp file
#1158 1154 feature add unit test authz
#1157 Add scripts for locating globus and gss api libraries
#1134 1114 add web unit testing to ci
#1163 Remove line split
#1153 Upgrade dependency of curl to 8.11
#1151 Throw error if variable is not defined properly
#1148 Provide default results for calls to gcs that allow proper evaluation later.
#1131 1130 bug entrypoint error ignoring errors
#1139 1128 Fix GCS Container json permission issues
#1136 Fix parallel runs, by adding random sleep
#1133 Add check for running container
#1129 1121 fix undefined build var in dockerfile
#1125 Remove overwrite of PATH
#1072 1071 add unit testing to web
#1112 Removed --no-cache flag
#1108 Update web version, with path due to change from CommonJS to ESM
#1079 1077 refactor support esm deprecate commonjs syntax
#1068 1065 fix ci cleanup script
#1064 1056 foxx test script fix
#1104 Fix parallel builds of retag and container build
#1092 Cleanup echo statements and make image tags explicit
#1094 make harbor script more robust to error response returned by api
#1101 Fix formatting of MessageLib and VersionUtils
#1098 Switch from using latest to CI_COMMIT_SHA
#1089 1086 bug python client pypi version
#1062 1055 fix foxx setup script
#1096 1057 trigger running foxx tests as part of ci
#1059 1057 trigger running foxx tests as part of ci
#1088 Add documentation about running DataFed Python package from source repo
#1081 Simplify cmake command
#1078 move flag to the correct position after ps
#1076 grab log output even if container fails
#1074 1073 add missing deployment key to node setup job
#1061 Add log output from ci end-to-end arango-setup job
#1054 1053 enable foxx tests in ci pipeline

DataFed-v2025.3.12.15.01

Pre-release

MAJOR Breaking changes

MINOR Feature

1. [987] - This implements serialized GitLab CI pipelines
2. [918] - Add admin script for making a user an admin
3. [1009] - Add collections to database for holding Globus tokens.
4. [1002] - Add backoff for task worker when database fails to prevent log overflow.
5. [1109] - Add support for Globus token association to Globus collection and user.
6. [1215] - Add support for fetching Globus tokens associated with Globus collection and user.
7. [1214] - Add support for mapped collections on transfer tasks

PATCH Bug fixes/Technical Debt/Documentation

1. [984] - Fixes {server_default} from showing up in path.
2. [990] - Will stop running containers so that we can prune them.
3. [988] - Removed non-working subscribe function.
4. [958] - Addresses issues raised by static code analyzer.
5. [962] - Adds script that will check that docker image is in registry
6. [1015] - Uses abs path to ci pipeline script for gcs build jobs
7. [1013] - add set +e and set -e around is active check for arango service
8. [1011] - Add python script and ci job to clean up GCS node keys.
9. [1023] - Address unbound variables in harbor ci script.
10. [1027] - Fix clear_db.sh script by replacing console.log with print
11. [1012] - Allow customized base build image for Docker dependencies and runtime Dockerfiles
12. [986] - Design improvement to upload and download transfer box.
13. [985] - Handles longer than needed timeouts on ui pages.
14. [1053] - Set CMake to enable foxx tests when built in the CI.
15. [1086] - Address XML-RPC bug from deprecated client
16. [1149] - Docker container GCS Collection Mount Bug Fix
17. [1168] - Add authz unit testing to the CI
18. [1200] - Add JavaScript linter (eslint) and (prettier) formatter for JavaScript
19. [1180] - Refactor of authz foxx module, split into objects and added unit tests
20. [1223] - Fix deprecated method usage for Protobuf in Python client library
21. [1257] - Bug in authz library parsing of authz.conf file, globus paths incorrectly sanitized when using just '/'
22. [1255] - Fixes bug, in libauthz with log_path var not being read in from config file.
23. [1268] - Bug fixes unbound variable in foxx entrypoint file.
24. [1269] - Update Arangodb to 3.12.4
25. [1288] - Bug Jupyter Notebook in documentation were fixed.
26. [1321] - Refactor, allow core config threads via env vars

DataFed-v2025.2.12.14.00-RC

Pre-release

MAJOR Breaking changes

MINOR Feature

1. #1240 [DLT-1110] Mapped Collection Endpoint Browse (1/4)
2. #1231 Revert "[DLT-1110] Mapped Collection Endpoint Browse (1/4)"
3. #1228 Feature DLT 1120 retrieve transfer token
4. #1224 [DAPS, Foxx] - 1180 refactor Part 2 authz
5. #1218 1216 daps feature add improved ci scanning
6. #1217 Feature DLT 1127 relay new token data
7. #1210 1209 feature add jsdoc linter
8. #1196 Add support for authz lookup
9. #1194 Feature DLT 1109 foxx new token data
10. #1192 Add DataFed dependency paths to curl so SSL will be found
11. #1160 1106 updated proto messages for setting token
12. #1141 DLT 1119 Provide method for Globus Authorize URL

PATCH Bug fixes/Technical Debt/Documentation

1. #1298 Fix bug in CI scripts associated with repo and gcs image build
2. #1239 Applied fix to deprecation warning
3. #1289 Add github action for linting jupyter
4. #1286 Update the schema documentation to improve the usability.
5. #1291 Update python notebooks
6. #1270 Add env var
7. #1271 Update Arangodb packaged version to using 3.12.4
8. #1274 Fix confusion in key uses in end to end tests
9. #1260 1255 feature mock core server initial integration testing authz
10. #1262 Address JSDoc complaints
11. #1258 Fix edge case in GridFTP Authz where '/' is used
12. #1251 Bug daps 1243 datafed web logging
13. #1238 Throw error if OpenStack error code returned from API
14. #1234 Fix bug on task_router.js abort function
15. #1227 Update nlhoman json version
16. #1221 1219 daps bug gitlab ci gcs log
17. #1212 1211 fix jsdoc errors
18. #1206 1205 refactor formatting across javascript
19. #1201 1200 feature GitHub action javascript linter formatter
20. #1197 Refactor foxx unit tests so they will run correctly
21. #1203 Address eslint errors
22. #1188 Remove --dump flag
23. #1187 1184 bug add set e foxx install
24. #1186 Changes to allow for and auto generate debug log level
25. #1181 1180 refactor foxx authz
26. #1175 Add -e back to core/database/test/scripts
27. #1169 1168 feature add authz unit tests to ci
28. #1167 Add missing comment on env variable
29. #1161 1159 refactor authz cpp file
30. #1158 1154 feature add unit test authz
31. #1157 Add scripts for locating globus and gss api libraries
32. #1134 1114 add web unit testing to ci
33. #1163 Remove line split
34. #1153 Upgrade dependency of curl to 8.11
35. #1151 Throw error if variable is not defined properly
36. #1148 Provide default results for calls to gcs that allow proper evaluation later.
37. #1131 1130 bug entrypoint error ignoring errors
38. #1139 1128 Fix GCS Container json permission issues
39. #1136 Fix parallel runs, by adding random sleep
40. #1133 Add check for running container
41. #1129 1121 fix undefined build var in dockerfile
42. #1125 Remove overwrite of PATH
43. #1072 1071 add unit testing to web
44. #1112 Removed --no-cache flag
45. #1108 Update web version, with path due to change from CommonJS to ESM
46. #1079 1077 refactor support esm deprecate commonjs syntax
47. #1068 1065 fix ci cleanup script
48. #1064 1056 foxx test script fix
49. #1104 Fix parallel builds of retag and container build
50. #1092 Cleanup echo statements and make image tags explicit
51. #1094 make harbor script more robust to error response returned by api
52. #1101 Fix formatting of MessageLib and VersionUtils
53. #1098 Switch from using latest to CI_COMMIT_SHA
54. #1089 1086 bug python client pypi version
55. #1062 1055 fix foxx setup script
56. #1096 1057 trigger running foxx tests as part of ci
57. #1059 1057 trigger running foxx tests as part of ci
58. #1088 Add documentation about running DataFed Python package from source repo
59. #1081 Simplify cmake command
60. #1078 move flag to the correct position after ps
61. #1076 grab log output even if container fails
62. #1074 1073 add missing deployment key to node setup job
63. #1061 Add log output from ci end-to-end arango-setup job
64. #1054 1053 enable foxx tests in ci pipeline

DataFed-v2024.6.17.10.40

v2024.6.17.10.40

MAJOR Breaking changes

MINOR Feature

1. [912] - Adds initial compose file and docker files for python-client
2. [909] - Added Support for Google Analytics
3. [916] - Reworked Docker build process to be faster and produce simpler and
   smaller images
4. [912] - Adding working compose instance for core metadata services.
5. [937] - Working metadata services running together as part of CI
6. [946] - Added docker compose for DataFed Repository and for Metadata Services
7. [955] - Adds repo pieces to CI with working end-to-end tests
8. [973] - Adds log output at the end of CI test pipeline
9. [968] - Adds the ability to specify both base and root path when creating
   Globus collections.
10. [1003] - Updated admin install documentation.
11. [1005] - Consolidate Development Environment setup documentation.
12. [970] - Adds Web UI testing to the CI pipeline.

PATCH Bug fixes/Technical Debt/Documentation

1. [914] - Improve GitHub template
2. [910] - Static code analysis and status checks fixed, improvements to CI
3. [923] - Fixed missing flag in certificate refresh script
4. [917] - Add additional files to .gitignore
5. [915] - Refactor CI to use pipelines Gitlab feature along with pipelines
6. [927] - Add vim swp files to .gitignore
7. [935] - Fixed CI branching with dynamic children, swiched to Harbor registry.
   Moved Blake's files into the correct folders.
8. [924] - Fix log messaging format in web server
9. [948] - Bug in getProjectId fixed, better handling of env variables in
   python.
10. [952] - Bug in globus cleanup python script fixed, was trying to use projects
    instead of the auth_client.
11. [953] - Refactored docker compose build scripts to address technical debt
12. [957] - Updated ArangoDB version used in compose file.
13. [956] - Adds improved error message, when paths of repo don't align.
14. [966] - CI refactor check that containers exist if not force build for branch.
15. [968] - Fixes bug by creating distinction between base and root path.
16. [981] - Fixes html injection that can occur from user name when displaying owner in schema dlg box.
17. [983] - Fixes google analytics by adding nonce which was broken.
18. [995] - Fixes issue with project and user folders in repo being created under root user permissions.
19. [994] - Fixes issue with spaces not being preserved in shell scripts from docker compose .env file.
20. [996] - Fixes bug in lego install script where function name had additional s
21. [998] - Fixing missing :latest tag on push in container, in common.yml of ci files
22. [999] - Fixes repo service entrypoint script to append to log file instead of rewriting

DataFed-v2023.8.21.10.40

First major release after refactoring.