Skip to content

Commit 1774623

Browse files
franzpoeschelax3l
franzpoeschel
and
ax3l
authored
Deprecate ADIOS1 backend (#1314)
* Add deprecation warning * Turn ADIOS1 off by default * Print warning on rank 0 only, make suppressible via OPENPMD_WARNING_SUPPRESS_ADIOS1_DEPRECATED=1 * Env Option: Rename & Document * CI: Cleanup * Add missing MPI_Init in test * Move parallel test to parallel tests * Docs: Update ADIOS1 Build Values * Docs: Two More Deprecated Hints Co-authored-by: Axel Huebl <axel.huebl@plasma.ninja>
1 parent 351d751 commit 1774623

File tree

9 files changed

+114
-76
lines changed

9 files changed

+114
-76
lines changed

‎.github/workflows/linux.yml‎

Lines changed: 4 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -82,14 +82,14 @@ jobs:
8282
export OPENPMD_BP_BACKEND=ADIOS1
8383
ctest --test-dir build --output-on-failure
8484
85-
clang7_nopy_ompi_h5_ad1_ad2_newLayout:
85+
clang7_nopy_ompi_h5_ad2_newLayout:
8686
runs-on: ubuntu-20.04
8787
if: github.event.pull_request.draft == false
8888
steps:
8989
- uses: actions/checkout@v2
9090
- name: Spack Cache
9191
uses: actions/cache@v2
92-
with: {path: /opt/spack, key: clang7_nopy_ompi_h5_ad1_ad2_v2 }
92+
with: {path: /opt/spack, key: clang7_nopy_ompi_h5_ad2_v2 }
9393
- name: Install
9494
run: |
9595
sudo apt-get update
@@ -237,7 +237,7 @@ jobs:
237237
cmake --build build --parallel 2
238238
ctest --test-dir build --output-on-failure
239239
240-
gcc9_py38_pd_nompi_h5_ad1_ad2_libcpp:
240+
gcc9_py38_pd_nompi_h5_ad2_libcpp:
241241
runs-on: ubuntu-20.04
242242
if: github.event.pull_request.draft == false
243243
steps:
@@ -246,7 +246,7 @@ jobs:
246246
run: |
247247
sudo apt-get update
248248
sudo apt-get install g++ libopenmpi-dev libhdf5-openmpi-dev libadios-dev python3 python3-numpy python3-mpi4py python3-pandas
249-
# TODO ADIOS1 (.pc file broken?) ADIOS2
249+
# TODO ADIOS2
250250
- name: Build
251251
env: {CXXFLAGS: -Werror, PKG_CONFIG_PATH: /usr/lib/x86_64-linux-gnu/pkgconfig}
252252
run: |

‎CMakeLists.txt‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -142,7 +142,7 @@ endfunction()
142142

143143
openpmd_option(MPI "Parallel, Multi-Node I/O for clusters" AUTO)
144144
openpmd_option(HDF5 "HDF5 backend (.h5 files)" AUTO)
145-
openpmd_option(ADIOS1 "ADIOS1 backend (.bp files)" AUTO)
145+
openpmd_option(ADIOS1 "ADIOS1 backend (.bp files)" OFF)
146146
openpmd_option(ADIOS2 "ADIOS2 backend (.bp files)" AUTO)
147147
openpmd_option(PYTHON "Enable Python bindings" AUTO)
148148

‎README.md‎

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -111,7 +111,7 @@ Shipped internally in `share/openPMD/thirdParty/`:
111111
I/O backends:
112112
* [JSON](https://en.wikipedia.org/wiki/JSON)
113113
* [HDF5](https://support.hdfgroup.org/HDF5) 1.8.13+ (optional)
114-
* [ADIOS1](https://www.olcf.ornl.gov/center-projects/adios) 1.13.1+ (optional)
114+
* [ADIOS1](https://www.olcf.ornl.gov/center-projects/adios) 1.13.1+ (optional, deprecated)
115115
* [ADIOS2](https://github.com/ornladios/ADIOS2) 2.7.0+ (optional)
116116

117117
while those can be built either with or without:
@@ -254,7 +254,7 @@ CMake controls options with prefixed `-D`, e.g. `-DopenPMD_USE_MPI=OFF`:
254254
|------------------------------|------------------|------------------------------------------------------------------------------|
255255
| `openPMD_USE_MPI` | **AUTO**/ON/OFF | Parallel, Multi-Node I/O for clusters |
256256
| `openPMD_USE_HDF5` | **AUTO**/ON/OFF | HDF5 backend (`.h5` files) |
257-
| `openPMD_USE_ADIOS1` | **AUTO**/ON/OFF | ADIOS1 backend (`.bp` files up to version BP3) |
257+
| `openPMD_USE_ADIOS1` | AUTO/ON/**OFF** | ADIOS1 backend (`.bp` files up to version BP3) - deprecated |
258258
| `openPMD_USE_ADIOS2` | **AUTO**/ON/OFF | ADIOS2 backend (`.bp` files in BP3, BP4 or higher) |
259259
| `openPMD_USE_PYTHON` | **AUTO**/ON/OFF | Enable Python bindings |
260260
| `openPMD_USE_INVASIVE_TESTS` | ON/**OFF** | Enable unit tests that modify source code <sup>1</sup> |

‎docs/source/backends/adios1.rst‎

Lines changed: 9 additions & 8 deletions
Original file line numberDiff line numberDiff line change
@@ -27,14 +27,15 @@ Backend-Specific Controls
2727
The following environment variables control ADIOS1 I/O behavior at runtime.
2828
Fine-tuning these is especially useful when running at large scale.
2929

30-
==================================== ========== ================================================================================
31-
environment variable default description
32-
==================================== ========== ================================================================================
33-
``OPENPMD_ADIOS_NUM_AGGREGATORS`` ``1`` Number of I/O aggregator nodes for ADIOS1 ``MPI_AGGREGATE`` transport method.
34-
``OPENPMD_ADIOS_NUM_OST`` ``0`` Number of I/O OSTs for ADIOS1 ``MPI_AGGREGATE`` transport method.
35-
``OPENPMD_ADIOS_HAVE_METADATA_FILE`` ``1`` Online creation of the adios journal file (``1``: yes, ``0``: no).
36-
``OPENPMD_BP_BACKEND`` ``ADIOS2`` Chose preferred ``.bp`` file backend if ``ADIOS1`` and ``ADIOS2`` are available.
37-
==================================== ========== ================================================================================
30+
============================================== ========== ================================================================================
31+
environment variable default description
32+
============================================== ========== ================================================================================
33+
``OPENPMD_ADIOS_NUM_AGGREGATORS`` ``1`` Number of I/O aggregator nodes for ADIOS1 ``MPI_AGGREGATE`` transport method.
34+
``OPENPMD_ADIOS_NUM_OST`` ``0`` Number of I/O OSTs for ADIOS1 ``MPI_AGGREGATE`` transport method.
35+
``OPENPMD_ADIOS_HAVE_METADATA_FILE`` ``1`` Online creation of the adios journal file (``1``: yes, ``0``: no).
36+
``OPENPMD_BP_BACKEND`` ``ADIOS2`` Chose preferred ``.bp`` file backend if ``ADIOS1`` and ``ADIOS2`` are available.
37+
``OPENPMD_ADIOS_SUPPRESS_DEPRECATED_WARNING`` ``0`` Set to ``1`` to suppress ADIOS1 deprecation warnings.
38+
============================================== ========== ================================================================================
3839

3940
Please refer to the `ADIOS1 manual, section 6.1.5 <https://users.nccs.gov/~pnorbert/ADIOS-UsersManual-1.13.1.pdf>`_ for details on I/O tuning.
4041

‎docs/source/dev/buildoptions.rst‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -14,7 +14,7 @@ CMake Option Values Description
1414
============================== =============== ========================================================================
1515
``openPMD_USE_MPI`` **AUTO**/ON/OFF Parallel, Multi-Node I/O for clusters
1616
``openPMD_USE_HDF5`` **AUTO**/ON/OFF HDF5 backend (``.h5`` files)
17-
``openPMD_USE_ADIOS1`` **AUTO**/ON/OFF ADIOS1 backend (``.bp`` files up to version BP3)
17+
``openPMD_USE_ADIOS1`` AUTO/ON/**OFF** ADIOS1 backend (``.bp`` files up to version BP3) - deprecated
1818
``openPMD_USE_ADIOS2`` **AUTO**/ON/OFF ADIOS2 backend (``.bp`` files in BP3, BP4 or higher)
1919
``openPMD_USE_PYTHON`` **AUTO**/ON/OFF Enable Python bindings
2020
``openPMD_USE_INVASIVE_TESTS`` ON/**OFF** Enable unit tests that modify source code :sup:`1`

‎docs/source/dev/dependencies.rst‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -27,7 +27,7 @@ Optional: I/O backends
2727

2828
* `JSON <https://en.wikipedia.org/wiki/JSON>`_
2929
* `HDF5 <https://support.hdfgroup.org/HDF5>`_ 1.8.13+
30-
* `ADIOS1 <https://www.olcf.ornl.gov/center-projects/adios>`_ 1.13.1+
30+
* `ADIOS1 <https://www.olcf.ornl.gov/center-projects/adios>`_ 1.13.1+ (deprecated)
3131
* `ADIOS2 <https://github.com/ornladios/ADIOS2>`_ 2.7.0+
3232

3333
while those can be build either with or without:

‎src/IO/AbstractIOHandlerHelper.cpp‎

Lines changed: 36 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -20,6 +20,8 @@
2020
*/
2121
#include "openPMD/IO/AbstractIOHandlerHelper.hpp"
2222

23+
#include "openPMD/config.hpp"
24+
2325
#include "openPMD/Error.hpp"
2426
#include "openPMD/IO/ADIOS/ADIOS1IOHandler.hpp"
2527
#include "openPMD/IO/ADIOS/ADIOS2IOHandler.hpp"
@@ -28,8 +30,13 @@
2830
#include "openPMD/IO/HDF5/HDF5IOHandler.hpp"
2931
#include "openPMD/IO/HDF5/ParallelHDF5IOHandler.hpp"
3032
#include "openPMD/IO/JSON/JSONIOHandler.hpp"
33+
#include "openPMD/auxiliary/Environment.hpp"
3134
#include "openPMD/auxiliary/JSON_internal.hpp"
3235

36+
#if openPMD_HAVE_MPI
37+
#include <mpi.h>
38+
#endif
39+
3340
#include <memory>
3441
#include <utility>
3542

@@ -55,6 +62,22 @@ namespace
5562
}
5663
throw "Unreachable";
5764
}
65+
66+
constexpr char const *adios1Deprecation = R"(
67+
[Deprecation warning]
68+
Development on the ADIOS1 IO library has ceased.
69+
Support for ADIOS1 in the openPMD-api has been deprecated
70+
and will be removed in a future version.
71+
72+
Please consider switching to ADIOS2.
73+
We recommend checking your ADIOS1 datasets for compatibility with ADIOS2.
74+
Conversion of data from one backend to another may optionally be achieved
75+
by using the `openpmd-pipe` tool.)
76+
77+
Suppress this warning via `export OPENPMD_ADIOS_SUPPRESS_DEPRECATED_WARNING=1`.)";
78+
79+
constexpr char const *suppressAdios1DeprecationWarning =
80+
"OPENPMD_ADIOS_SUPPRESS_DEPRECATED_WARNING";
5881
} // namespace
5982

6083
#if openPMD_HAVE_MPI
@@ -75,6 +98,15 @@ std::unique_ptr<AbstractIOHandler> createIOHandler<json::TracingJSON>(
7598
return constructIOHandler<ParallelHDF5IOHandler, openPMD_HAVE_HDF5>(
7699
"HDF5", path, access, comm, std::move(options));
77100
case Format::ADIOS1:
101+
if (auxiliary::getEnvNum(suppressAdios1DeprecationWarning, 0) == 0)
102+
{
103+
int rank;
104+
MPI_Comm_rank(comm, &rank);
105+
if (rank == 0)
106+
{
107+
std::cerr << adios1Deprecation << std::endl;
108+
}
109+
}
78110
return constructIOHandler<ParallelADIOS1IOHandler, openPMD_HAVE_ADIOS1>(
79111
"ADIOS1", path, access, std::move(options), comm);
80112
case Format::ADIOS2_BP:
@@ -144,6 +176,10 @@ std::unique_ptr<AbstractIOHandler> createIOHandler<json::TracingJSON>(
144176
return constructIOHandler<HDF5IOHandler, openPMD_HAVE_HDF5>(
145177
"HDF5", path, access, std::move(options));
146178
case Format::ADIOS1:
179+
if (auxiliary::getEnvNum(suppressAdios1DeprecationWarning, 0) == 0)
180+
{
181+
std::cerr << adios1Deprecation << std::endl;
182+
}
147183
return constructIOHandler<ADIOS1IOHandler, openPMD_HAVE_ADIOS1>(
148184
"ADIOS1", path, access, std::move(options));
149185
case Format::ADIOS2_BP:

‎test/CoreTest.cpp‎

Lines changed: 0 additions & 59 deletions
Original file line numberDiff line numberDiff line change
@@ -1354,63 +1354,4 @@ TEST_CASE("unavailable_backend", "[core]")
13541354
"'HDF5'.");
13551355
}
13561356
#endif
1357-
1358-
#if openPMD_HAVE_MPI
1359-
#if !openPMD_HAVE_ADIOS1
1360-
{
1361-
auto fail = []() {
1362-
Series(
1363-
"unavailable.bp",
1364-
Access::CREATE,
1365-
MPI_COMM_WORLD,
1366-
R"({"backend": "ADIOS1"})");
1367-
};
1368-
REQUIRE_THROWS_WITH(
1369-
fail(),
1370-
"Wrong API usage: openPMD-api built without support for backend "
1371-
"'ADIOS1'.");
1372-
}
1373-
#endif
1374-
#if !openPMD_HAVE_ADIOS2
1375-
{
1376-
auto fail = []() {
1377-
Series(
1378-
"unavailable.bp",
1379-
Access::CREATE,
1380-
MPI_COMM_WORLD,
1381-
R"({"backend": "ADIOS2"})");
1382-
};
1383-
REQUIRE_THROWS_WITH(
1384-
fail(),
1385-
"Wrong API usage: openPMD-api built without support for backend "
1386-
"'ADIOS2'.");
1387-
}
1388-
#endif
1389-
#if !openPMD_HAVE_ADIOS1 && !openPMD_HAVE_ADIOS2
1390-
{
1391-
auto fail = []() {
1392-
Series("unavailable.bp", Access::CREATE, MPI_COMM_WORLD);
1393-
};
1394-
REQUIRE_THROWS_WITH(
1395-
fail(),
1396-
"Wrong API usage: openPMD-api built without support for backend "
1397-
"'ADIOS2'.");
1398-
}
1399-
#endif
1400-
#if !openPMD_HAVE_HDF5
1401-
{
1402-
auto fail = []() {
1403-
Series(
1404-
"unavailable.h5",
1405-
Access::CREATE,
1406-
MPI_COMM_WORLD,
1407-
R"({"backend": "HDF5"})");
1408-
};
1409-
REQUIRE_THROWS_WITH(
1410-
fail(),
1411-
"Wrong API usage: openPMD-api built without support for backend "
1412-
"'HDF5'.");
1413-
}
1414-
#endif
1415-
#endif
14161357
}

‎test/ParallelIOTest.cpp‎

Lines changed: 60 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1604,4 +1604,64 @@ TEST_CASE("append_mode", "[parallel]")
16041604
}
16051605
}
16061606
}
1607+
1608+
TEST_CASE("unavailable_backend", "[core][parallel]")
1609+
{
1610+
#if !openPMD_HAVE_ADIOS1
1611+
{
1612+
auto fail = []() {
1613+
Series(
1614+
"unavailable.bp",
1615+
Access::CREATE,
1616+
MPI_COMM_WORLD,
1617+
R"({"backend": "ADIOS1"})");
1618+
};
1619+
REQUIRE_THROWS_WITH(
1620+
fail(),
1621+
"Wrong API usage: openPMD-api built without support for backend "
1622+
"'ADIOS1'.");
1623+
}
1624+
#endif
1625+
#if !openPMD_HAVE_ADIOS2
1626+
{
1627+
auto fail = []() {
1628+
Series(
1629+
"unavailable.bp",
1630+
Access::CREATE,
1631+
MPI_COMM_WORLD,
1632+
R"({"backend": "ADIOS2"})");
1633+
};
1634+
REQUIRE_THROWS_WITH(
1635+
fail(),
1636+
"Wrong API usage: openPMD-api built without support for backend "
1637+
"'ADIOS2'.");
1638+
}
1639+
#endif
1640+
#if !openPMD_HAVE_ADIOS1 && !openPMD_HAVE_ADIOS2
1641+
{
1642+
auto fail = []() {
1643+
Series("unavailable.bp", Access::CREATE, MPI_COMM_WORLD);
1644+
};
1645+
REQUIRE_THROWS_WITH(
1646+
fail(),
1647+
"Wrong API usage: openPMD-api built without support for backend "
1648+
"'ADIOS2'.");
1649+
}
1650+
#endif
1651+
#if !openPMD_HAVE_HDF5
1652+
{
1653+
auto fail = []() {
1654+
Series(
1655+
"unavailable.h5",
1656+
Access::CREATE,
1657+
MPI_COMM_WORLD,
1658+
R"({"backend": "HDF5"})");
1659+
};
1660+
REQUIRE_THROWS_WITH(
1661+
fail(),
1662+
"Wrong API usage: openPMD-api built without support for backend "
1663+
"'HDF5'.");
1664+
}
1665+
#endif
1666+
}
16071667
#endif

0 commit comments

Comments
 (0)