Release: 0.16.1 (#1676)

* Release: 0.16.1

Signed-off-by: Axel Huebl <[email protected]>

* Fix new_version.py

* Update version in windows.yml

* Reformat with yapf3

* Fix greedy Regex parsing

* Changelog: Add PRs #1681 #1679 #1678

* Fix: CMake Internal Control (#1678)

The `openPMD_USE_INTERNAL_*` options are for general internal/external
control. They were accidentally overwritten by new tar flags
and sometimes incomplete.

Signed-off-by: Axel Huebl <[email protected]>

* Revert "CMake Tar: TOML11 3.7.1 (#1673)" (#1679)

This reverts commit aea4b72.

* Replace deprecated makeSuite with loadTestsFromTestCase (#1681)

Removed with Python 3.13

* Doc: Fix Shipped Internally (#1682)

Downloaded now.

Signed-off-by: Axel Huebl <[email protected]>

* `PYBIND11_FINDPYTHON=ON` (#1684)

Reuse our `find_package(Python ...)` call and use new CMake logic in
pybind11.
https://pybind11.readthedocs.io/en/stable/compiling.html#modules-with-cmake
https://cmake.org/cmake/help/latest/command/find_package.html#config-mode-version-selection

Signed-off-by: Axel Huebl <[email protected]>

* Fix ICX build (#1690)

Error was:
 /home/runner/work/openPMD-api/openPMD-api/include/openPMD/backend/Container.hpp:262:32:
error: no member named 'm_container' in 'Container<T, T_key,
T_container>'
  262 |         container().swap(other.m_container);

* Fix Leftover ADIOS1 Mentions (#1683)

* Fix Leftover ADIOS1 Mentions

Signed-off-by: Axel Huebl <[email protected]>

* Remove ADIOS1 from examples

---------

Signed-off-by: Axel Huebl <[email protected]>
Co-authored-by: Franz Pöschel <[email protected]>

* Explicitly specify ADIOS2 components in openPMDConfig.cmake (#1693)

* Explicitly specify ADIOS2 components in openPMDConfig.cmake

When including openPMD in downstream CMake and that CMake script
contains Fortran, ADIOS2 will automatically detect that. But since we
do not reexport the ADIOS2 headers, we only need the components that
openPMD explicitly uses: CXX and MPI.

* Same in CMakeLists.txt

* Remove mpirun_workaround.sh (#1698)

Seems to no longer be necessary in the affected CI runs

* Properly check for empty HDF5_VERSION in CMake (#1702)

The old test apparently invoked some weird CMake legacy behavior.
This should make the pinned HDF5 Conda version unnecessary (#1701) since
the version with missing HDF5_VERSION is correctly dealt with.

* HDF5: Delete and re-create attribute when overwriting with diff. type (#1697)

* HDF5: Delete and re-create attribute when overwriting with diff. type

* WIP: Change datatype in test

Somehow truncation test is broken by this

* Add forgotten H5Aclose() call

* TOUCH IOTask: Avoid setting files as dirty in non-write modes (#1704)

* Bugfix: Consider dirty upon touch() only in write modes

* Add error check to avoid writing in read-only mode

* Changelog

* Use ULONG for particle patches

* Upgrade appleclang14 build to MacOS 13

> The macOS-12 environment is deprecated, consider switching to macOS-13,
macOS-14 (macos-latest) or macOS-15. For more details, see
actions/runner-images#10721

* Update Changelog

* Release Date

---------

Signed-off-by: Axel Huebl <[email protected]>
Co-authored-by: Franz Pöschel <[email protected]>

Author: ax3l

.github/workflows/linux.yml

@@ -328,8 +328,7 @@ jobs:
           -DopenPMD_USE_MPI=ON    \
           -DopenPMD_USE_HDF5=ON   \
           -DopenPMD_USE_ADIOS2=ON \
-          -DopenPMD_USE_INVASIVE_TESTS=ON \
-          -DMPIEXEC_EXECUTABLE=".github/workflows/mpirun_workaround.sh"
+          -DopenPMD_USE_INVASIVE_TESTS=ON
         cmake --build build --parallel 2
         cd build
         ctest --output-on-failure

.github/workflows/macos.yml

@@ -40,13 +40,12 @@ jobs:
           -DopenPMD_USE_MPI=ON    \
           -DopenPMD_USE_HDF5=ON   \
           -DopenPMD_USE_ADIOS2=ON \
-          -DopenPMD_USE_INVASIVE_TESTS=ON \
-          -DMPIEXEC_EXECUTABLE=".github/workflows/mpirun_workaround.sh"
+          -DopenPMD_USE_INVASIVE_TESTS=ON
         cmake --build build --parallel 3
         ctest --test-dir build --verbose
 
   appleclang14_py:
-    runs-on: macos-12
+    runs-on: macos-13
     if: github.event.pull_request.draft == false
     steps:
     - uses: actions/checkout@v4

.github/workflows/mpirun_workaround.sh

@@ -42,7 +42,7 @@ jobs:
 
         python3.exe -m pip wheel .
         if(!$?) { Exit $LASTEXITCODE }
-        python3.exe -m pip install openPMD_api-0.16.0.dev0-cp39-cp39-win_amd64.whl
+        python3.exe -m pip install openPMD_api-0.16.1-cp39-cp39-win_amd64.whl
         if(!$?) { Exit $LASTEXITCODE }
 
         python3.exe -c "import openpmd_api as api; print(api.variants)"

.github/workflows/windows.yml

@@ -3,6 +3,41 @@
 Changelog
 =========
 
+0.16.1
+------
+**Date:** 2025-01-15
+
+New Backends, Extension, Perf. & Memory
+
+This is the 0.16.0 release but with internal version bumps where they were missing.
+
+Changes to "0.16.0"
+^^^^^^^^^^^^^^^^^^^
+
+Bug Fixes
+"""""""""
+
+- HDF5: Delete and re-create attribute when overwriting with diff. type #1697
+- TOUCH IOTask: Avoid setting files as dirty in non-write modes #1704
+- Fix CMake variables for controlling internal dependencies #1678
+- Bump toml11 dependency to ``v4.2.0`` by default #1679
+- Set ``PYBIND11_FINDPYTHON=ON`` #1684
+- Properly check for empty ``HDF5_VERSION`` in CMake #1702
+- Fix ICX build #1690
+- Explicitly specify ADIOS2 components in openPMDConfig.cmake #1693
+- Example 12: Use ``ULONG`` for particle patches #1710
+
+Other
+"""""
+
+- Replace deprecated Python unittest API call #1681
+- Remove ``mpirun_workaround.sh`` #1698
+- CI: Upgrade appleclang14 build to MacOS 13 #1703
+- Docs:
+
+  - Fix Shipped Internally #1682
+  - Fix leftover ADIOS2 mentions #1683
+
 0.16.0
 ------
 **Date:** 2024-10-07

CHANGELOG.rst

@@ -37,7 +37,7 @@ contact:
   orcid: https://orcid.org/0000-0003-1943-7141
   email: [email protected]
 title: "openPMD-api: C++ & Python API for Scientific I/O with openPMD"
-version: 0.16.0-dev
+version: 0.16.1
 repository-code: https://github.com/openPMD/openPMD-api
 doi: 10.14278/rodare.27
 license: LGPL-3.0-or-later

CITATION.cff

@@ -2,7 +2,7 @@
 #
 cmake_minimum_required(VERSION 3.22.0)
 
-project(openPMD VERSION 0.16.0) # LANGUAGES CXX
+project(openPMD VERSION 0.16.1) # LANGUAGES CXX
 
 # the openPMD "markup"/"schema" standard version
 set(openPMD_STANDARD_VERSION 1.1.0)
@@ -314,7 +314,7 @@ endif()
 string(CONCAT openPMD_HDF5_STATUS "")
 # version: lower limit
 if(openPMD_HAVE_HDF5)
-    if(HDF5_VERSION STREQUAL "")
+    if("${HDF5_VERSION}" STREQUAL "")
         message(WARNING "HDF5_VERSION is empty. Now assuming it is 1.8.13 or newer.")
     else()
         if(HDF5_VERSION VERSION_LESS 1.8.13)
@@ -360,20 +360,24 @@ if(openPMD_HDF5_STATUS)
 endif()
 
 # external library: ADIOS2 (optional)
+set(openPMD_REQUIRED_ADIOS2_COMPONENTS CXX)
+if(openPMD_HAVE_MPI)
+    list(APPEND openPMD_REQUIRED_ADIOS2_COMPONENTS MPI)
+endif()
 if(openPMD_USE_ADIOS2 STREQUAL AUTO)
-    find_package(ADIOS2 2.7.0 CONFIG)
+    find_package(ADIOS2 2.7.0 CONFIG COMPONENTS ${openPMD_REQUIRED_ADIOS2_COMPONENTS})
     if(ADIOS2_FOUND)
         set(openPMD_HAVE_ADIOS2 TRUE)
     else()
         set(openPMD_HAVE_ADIOS2 FALSE)
     endif()
 elseif(openPMD_USE_ADIOS2)
-    find_package(ADIOS2 2.7.0 REQUIRED CONFIG)
+    find_package(ADIOS2 2.7.0 REQUIRED CONFIG COMPONENTS ${openPMD_REQUIRED_ADIOS2_COMPONENTS})
     set(openPMD_HAVE_ADIOS2 TRUE)
 else()
     set(openPMD_HAVE_ADIOS2 FALSE)
 endif()
-# TODO: Check if ADIOS2 is parallel when openPMD_HAVE_MPI is ON
+unset(openPMD_REQUIRED_ADIOS2_COMPONENTS)
 
 # external library: pybind11 (optional)
 include(${openPMD_SOURCE_DIR}/cmake/dependencies/pybind11.cmake)

CMakeLists.txt

@@ -99,7 +99,7 @@ Required:
 * CMake 3.22.0+
 * C++17 capable compiler, e.g., g++ 7+, clang 7+, MSVC 19.15+, icpc 19+, icpx
 
-Shipped internally in `share/openPMD/thirdParty/`:
+Shipped internally (downloaded by CMake unless `openPMD_SUPERBUILD=OFF` is set):
 * [Catch2](https://github.com/catchorg/Catch2) 2.13.10+ ([BSL-1.0](https://github.com/catchorg/Catch2/blob/master/LICENSE.txt))
 * [pybind11](https://github.com/pybind/pybind11) 2.13.0+ ([new BSD](https://github.com/pybind/pybind11/blob/master/LICENSE))
 * [NLohmann-JSON](https://github.com/nlohmann/json) 3.9.1+ ([MIT](https://github.com/nlohmann/json/blob/develop/LICENSE.MIT))
@@ -304,7 +304,7 @@ export CMAKE_PREFIX_PATH=$HOME/somepath:$CMAKE_PREFIX_PATH
 Use the following lines in your project's `CMakeLists.txt`:
 ```cmake
 # supports:                       COMPONENTS MPI NOMPI HDF5 ADIOS2
-find_package(openPMD 0.16.0 CONFIG)
+find_package(openPMD 0.16.1 CONFIG)
 
 if(openPMD_FOUND)
     target_link_libraries(YourTarget PRIVATE openPMD::openPMD)
@@ -332,7 +332,7 @@ set(openPMD_INSTALL OFF)            # or instead use:
 set(openPMD_USE_PYTHON OFF)
 FetchContent_Declare(openPMD
   GIT_REPOSITORY "https://github.com/openPMD/openPMD-api.git"
-  GIT_TAG        "0.16.0")
+  GIT_TAG        "0.16.1")
 FetchContent_MakeAvailable(openPMD)
 ```
 

README.md

@@ -1,49 +1,49 @@
 function(find_catch2)
     if(TARGET Catch2::Catch2)
         message(STATUS "Catch2::Catch2 target already imported")
-    elseif(openPMD_catch_src)
-        message(STATUS "Compiling local Catch2 ...")
-        message(STATUS "Catch2 source path: ${openPMD_catch_src}")
-        if(NOT IS_DIRECTORY ${openPMD_catch_src})
-            message(FATAL_ERROR "Specified directory openPMD_catch_src='${openPMD_catch_src}' does not exist!")
-        endif()
-    elseif(openPMD_catch_tar)
-        message(STATUS "Downloading Catch2 ...")
-        message(STATUS "Catch2 source: ${openPMD_catch_tar}")
     elseif(openPMD_USE_INTERNAL_CATCH)
-        message(STATUS "Downloading Catch2 ...")
-        message(STATUS "Catch2 repository: ${openPMD_catch_repo} (${openPMD_catch_branch})")
+        if(openPMD_catch_src)
+            message(STATUS "Compiling local Catch2 ...")
+            message(STATUS "Catch2 source path: ${openPMD_catch_src}")
+            if(NOT IS_DIRECTORY ${openPMD_catch_src})
+                message(FATAL_ERROR "Specified directory openPMD_catch_src='${openPMD_catch_src}' does not exist!")
+            endif()
+        elseif(openPMD_catch_tar)
+            message(STATUS "Downloading Catch2 ...")
+            message(STATUS "Catch2 source: ${openPMD_catch_tar}")
+        elseif(openPMD_catch_branch)
+            message(STATUS "Downloading Catch2 ...")
+            message(STATUS "Catch2 repository: ${openPMD_catch_repo} (${openPMD_catch_branch})")
+        endif()
     endif()
     if(TARGET Catch2::Catch2)
         # nothing to do, target already exists in the superbuild
-    elseif(openPMD_USE_INTERNAL_CATCH OR openPMD_catch_src OR openPMD_catch_tar)
-        if(openPMD_catch_src)
-            add_subdirectory(${openPMD_catch_src} _deps/localCatch2-build/)
+    elseif(openPMD_USE_INTERNAL_CATCH AND openPMD_catch_src)
+        add_subdirectory(${openPMD_catch_src} _deps/localCatch2-build/)
+    elseif(openPMD_USE_INTERNAL_CATCH AND (openPMD_catch_tar OR openPMD_catch_branch))
+        include(FetchContent)
+        if(openPMD_catch_tar)
+            FetchContent_Declare(fetchedCatch2
+                URL             ${openPMD_catch_tar}
+                URL_HASH        ${openPMD_catch_tar_hash}
+                BUILD_IN_SOURCE OFF
+            )
         else()
-            include(FetchContent)
-            if(openPMD_catch_tar)
-                FetchContent_Declare(fetchedCatch2
-                    URL             ${openPMD_catch_tar}
-                    URL_HASH        ${openPMD_catch_tar_hash}
-                    BUILD_IN_SOURCE OFF
-                )
-            else()
-                FetchContent_Declare(fetchedCatch2
-                    GIT_REPOSITORY ${openPMD_catch_repo}
-                    GIT_TAG        ${openPMD_catch_branch}
-                    BUILD_IN_SOURCE OFF
-                )
-            endif()
-            FetchContent_MakeAvailable(fetchedCatch2)
-
-            # advanced fetch options
-            mark_as_advanced(FETCHCONTENT_BASE_DIR)
-            mark_as_advanced(FETCHCONTENT_FULLY_DISCONNECTED)
-            mark_as_advanced(FETCHCONTENT_QUIET)
-            #mark_as_advanced(FETCHCONTENT_SOURCE_DIR_FETCHEDCatch2)
-            mark_as_advanced(FETCHCONTENT_UPDATES_DISCONNECTED)
-            #mark_as_advanced(FETCHCONTENT_UPDATES_DISCONNECTED_FETCHEDCatch2)
+            FetchContent_Declare(fetchedCatch2
+                GIT_REPOSITORY ${openPMD_catch_repo}
+                GIT_TAG        ${openPMD_catch_branch}
+                BUILD_IN_SOURCE OFF
+            )
         endif()
+        FetchContent_MakeAvailable(fetchedCatch2)
+
+        # advanced fetch options
+        mark_as_advanced(FETCHCONTENT_BASE_DIR)
+        mark_as_advanced(FETCHCONTENT_FULLY_DISCONNECTED)
+        mark_as_advanced(FETCHCONTENT_QUIET)
+        #mark_as_advanced(FETCHCONTENT_SOURCE_DIR_FETCHEDCatch2)
+        mark_as_advanced(FETCHCONTENT_UPDATES_DISCONNECTED)
+        #mark_as_advanced(FETCHCONTENT_UPDATES_DISCONNECTED_FETCHEDCatch2)
     elseif(NOT openPMD_USE_INTERNAL_CATCH)
         find_package(Catch2 2.13.10 CONFIG REQUIRED)
         message(STATUS "Catch2: Found version '${Catch2_VERSION}'")

cmake/dependencies/catch.cmake

@@ -1,49 +1,49 @@
 function(find_json)
     if(TARGET nlohmann_json::nlohmann_json)
         message(STATUS "nlohmann_json::nlohmann_json target already imported")
-    elseif(openPMD_json_src)
-        message(STATUS "Compiling local nlohmann_json ...")
-        message(STATUS "nlohmann_json source path: ${openPMD_json_src}")
-        if(NOT IS_DIRECTORY ${openPMD_json_src})
-            message(FATAL_ERROR "Specified directory openPMD_json_src='${openPMD_json_src}' does not exist!")
-        endif()
-    elseif(openPMD_json_tar)
-        message(STATUS "Downloading nlohmann_json ...")
-        message(STATUS "nlohmann_json source: ${openPMD_json_tar}")
     elseif(openPMD_USE_INTERNAL_JSON)
-        message(STATUS "Downloading nlohmann_json ...")
-        message(STATUS "nlohmann_json repository: ${openPMD_json_repo} (${openPMD_json_branch})")
+        if(openPMD_json_src)
+            message(STATUS "Compiling local nlohmann_json ...")
+            message(STATUS "nlohmann_json source path: ${openPMD_json_src}")
+            if(NOT IS_DIRECTORY ${openPMD_json_src})
+                message(FATAL_ERROR "Specified directory openPMD_json_src='${openPMD_json_src}' does not exist!")
+            endif()
+        elseif(openPMD_json_tar)
+            message(STATUS "Downloading nlohmann_json ...")
+            message(STATUS "nlohmann_json source: ${openPMD_json_tar}")
+        elseif(openPMD_json_branch)
+            message(STATUS "Downloading nlohmann_json ...")
+            message(STATUS "nlohmann_json repository: ${openPMD_json_repo} (${openPMD_json_branch})")
+        endif()
     endif()
     if(TARGET nlohmann_json::nlohmann_json)
         # nothing to do, target already exists in the superbuild
-    elseif(openPMD_USE_INTERNAL_JSON OR openPMD_json_src OR openPMD_json_tar)
-        if(openPMD_json_src)
-            add_subdirectory(${openPMD_json_src} _deps/localnlohmann_json-build/)
+    elseif(openPMD_USE_INTERNAL_JSON AND openPMD_json_src)
+        add_subdirectory(${openPMD_json_src} _deps/localnlohmann_json-build/)
+    elseif(openPMD_USE_INTERNAL_JSON AND (openPMD_json_tar OR openPMD_json_branch))
+        include(FetchContent)
+        if(openPMD_json_tar)
+            FetchContent_Declare(fetchednlohmann_json
+                URL             ${openPMD_json_tar}
+                URL_HASH        ${openPMD_json_tar_hash}
+                BUILD_IN_SOURCE OFF
+            )
         else()
-            include(FetchContent)
-            if(openPMD_json_tar)
-                FetchContent_Declare(fetchednlohmann_json
-                    URL             ${openPMD_json_tar}
-                    URL_HASH        ${openPMD_json_tar_hash}
-                    BUILD_IN_SOURCE OFF
-                )
-            else()
-                FetchContent_Declare(fetchednlohmann_json
-                    GIT_REPOSITORY ${openPMD_json_repo}
-                    GIT_TAG        ${openPMD_json_branch}
-                    BUILD_IN_SOURCE OFF
-                )
-            endif()
-            FetchContent_MakeAvailable(fetchednlohmann_json)
-
-            # advanced fetch options
-            mark_as_advanced(FETCHCONTENT_BASE_DIR)
-            mark_as_advanced(FETCHCONTENT_FULLY_DISCONNECTED)
-            mark_as_advanced(FETCHCONTENT_QUIET)
-            #mark_as_advanced(FETCHCONTENT_SOURCE_DIR_FETCHEDnlohmann_json)
-            mark_as_advanced(FETCHCONTENT_UPDATES_DISCONNECTED)
-            #mark_as_advanced(FETCHCONTENT_UPDATES_DISCONNECTED_FETCHEDnlohmann_json)
+            FetchContent_Declare(fetchednlohmann_json
+                GIT_REPOSITORY ${openPMD_json_repo}
+                GIT_TAG        ${openPMD_json_branch}
+                BUILD_IN_SOURCE OFF
+            )
         endif()
+        FetchContent_MakeAvailable(fetchednlohmann_json)
+
+        # advanced fetch options
+        mark_as_advanced(FETCHCONTENT_BASE_DIR)
+        mark_as_advanced(FETCHCONTENT_FULLY_DISCONNECTED)
+        mark_as_advanced(FETCHCONTENT_QUIET)
+        #mark_as_advanced(FETCHCONTENT_SOURCE_DIR_FETCHEDnlohmann_json)
+        mark_as_advanced(FETCHCONTENT_UPDATES_DISCONNECTED)
+        #mark_as_advanced(FETCHCONTENT_UPDATES_DISCONNECTED_FETCHEDnlohmann_json)
     elseif(NOT openPMD_USE_INTERNAL_JSON)
         find_package(nlohmann_json 3.9.1 CONFIG REQUIRED)
         message(STATUS "nlohmann_json: Found version '${nlohmann_json_VERSION}'")