Building OSTRICH

Instructions for building OSTRICH on various platforms

Initial Publication:

Modified:

Contents

1 Building on Linux

2.1.2 Visual Studio 2019

1.1 Parallel Build with MPI

2.2 Importing OSTRICH into Visual Studio

1.2 Serial build

2.3 Adding Dependencies into Visual Studio

1.3 Serial Build with Database Output

2.3.1 Microsoft MPI

2 Building on Windows

2.3.2 Microsoft ADO

2.1 Dependencies

2.4 Modifying Compilation Options

2.1.1 Microsoft MPI

2.5 Troubleshooting

1 Building on Linux

A user may need to build from source on Linux to ensure that all OSTRICH dependencies are correctly referenced. While OSTRICH is by and large self contained, there are limited external dependencies that will need to be satisfied for the toolkit to operate correctly. If a user makes modifications to the source code, OSTRICH will also need to be recompiled prior to use.

A makefile is included within the source directory to facilitate building on Linux. The makefile includes the three most common OSTRICH use configurations, and it should be sufficient for the majority of users. Users may need to add directories to the environment path if a non-standard Linux configuration is being utilized.

1.1 Parallel build with MPI

This should be the default build option a user should follow unless a specific need requires one of the other options. Building OSTRICH with MPI gives a user the option to start OSTRICH in parallel mode. This allows OSTRICH to take advantage of solving multiple models simultanteously, if the selected algorithm supports parallel solutions. However, the MPI enabled build can also be utilized in a serial configuration by specifying a single process. This flexibility allows a user to compile OSTRICH once on a given machine and use it across different scenarios.

The MPI build supports both Intel and GCC compilers, if available on a system. To compile MPI enabled OSTRICH using the GCC compiler stack, from within the make directory issue the following:

make GCC_MPI

To compile MPI enabled OSTRICH using the Intel compiler stack, from within the make directory issue the following:

make ICC_MPI

Both methods require an MPI installation to be available on the system path. If MPI is not enabled on the machine, OSTRICH will not compile.

1.2 Serial build

OSTRICH can also be built in serial mode. Models, even when a parallel algorithm is selected, will be solved sequentially one after the other. No speedup will therefore be available from using a parallelized algorithm. This can be helpful when trying to debug a model or when a model solves quickly to not require the performance benefits of parallelization.

The OSTRICH source code is configured for MPI and would not build in serial mode without a special provision. To support a serial build process, the makefile targets MPI stub files within the extern folder of the repository. This provides the required MPI functions to allow the code to compile which block OSTRICH from parallelizing those operations. The extern folder must therefore be provided along with the src folder for the build to be successful.

The serial build supports both Intel and GCC compilers, if available on a system. To compile serial OSTRICH using the GCC compiler stack, from within the make directory issue the following:

make GCC

To compile serial OSTRICH using the Intel compiler stack, from within the make directory issue the following:

make ICC

1.3 Serial build with database output

The third method for building OSTRICH is database mode. This utilizes file operations to simulate standard MPI operations. Like the serial build process, the makefile targets MPI stup files within the extern folder of the repository. This provides the required MPI functions to allow the code to compile which block OSTRICH from parallelizing those operations. The extern folder must therefore be provided along with the src folder for the build to be successful.

Although the database mode is available, this is by far the least used configuration of OSTRICH. If this mode is utilized, it is strongly suggested that the user perform a limited run or reduced test case to confirm that OSTRICH behaves as expected for the selected algorithm.

The database build supports both Intel and GCC compilers, if available on a system. To compile database OSTRICH using the GCC compiler stack, from within the make directory issue the following:

make GCC_DB

To compile database OSTRICH using the Intel compiler stack, from within the make directory issue the following:

make ICC_DB

2 Building on Windows

Building on Windows can be more challenging than Linux as additional configuration steps are necessary. Unless a custom build of OSTRICH is required, it is strongly recommended that an existing build of OSTRICH be utilized to save time and effort. With that caveat, it is straightforward to create a custom Windows build once the project is configured correctly. The following sections outline how to build OSTRICH for the Windows platform.

2.1 Dependencies

Installation of the necessary components and libraries will require administrative access on the Windows machine. If you do not have administrative authority on the machine, you will not be able to install all the necessary dependencies to build OSTRICH in the default locations. Checking that you have proper permissions now will reduce subsequent frustration during the build process.

The outlined method uses Microsoft development tools and compiler stack for the build process. There are numerous workflows that could build OSTRICH for the Windows platform. This approach has been found to be the most straightforward and most compatible with subsequent OSTRICH use cases. If your model intended for use within OSTRICH is built using a stack other than Microsoft (i.e. Cwgwin), be aware of that unexpected compatability issues may present themselves during the solution process.

2.1.1 Microsoft MPI

An installation of MPI is required to build and run OSTRICH. On Windows, the Microsoft MPI version is recommended. The installation for v10.0 can be found here. Make sure to install both components, the library and the SDK, from the download. Both will be required to build and run OSTRICH.

2.1.2 Visual Studio 2019

Visual Studio 2019 is utilized to provide both a compiler and IDE in which to structure project dependencies. The free Community version is sufficient, which can be downloaded from Microsoft directly.

Visual Studio does not install the C++ components by default. Additional components must be selected during the Visual Studio installation based on how the user will employ Visual Studio. For this case, make sure that the following components are selected:

  • C++/CLI support v142
  • MSVC v142 - VS 2019 C++ x64/x86 build tools
  • C++ core features
  • C++ ATL v142 (x86 and x64)
  • C++ MFC v142 (x86 and x64)
  • Windows 10 SDK
  • Windows Universal C Runtime

The download size for these components will be quite large. Make sure that you have a fast, stable internet connection to facilitate the download.

2.2 Importing OSTRICH into Visual Studio

Once Visual Studio is installed on your machine, the next steps are to create a Visual Studio solution for OSTRICH. A solution is how Visual Studio organizes related code. A solution can have multiple projects that are compiled together to create a final product. In this case, a solution with a single project will be utilized.

Open Visual Studio. The splash page will provide several options. Select "Create a new project". This will bring up an secondary page asking about the type of project. Type "C++" into the search, which will provide the option for an empty project. Select "Empty Project" and click "Next". This final page will ask for the name of the project and it's location. Put is something memorable that indicates the purpose of the project. The default is to name the solution the same as the project; however, you can change the solution naming convention at this time as well. Click "Create" to complete the process.

Visual Studio will now open a blank IDE window. The next step is to add the OSTRICH files into the project. This makes Visual Studio aware of the files and allows us to work with them within the IDE. If the Solution Explorer is not visible, add it from the View menu. Within the Solution Explorer, right click on the Header Files folder and choose "Add->Existing Item". This should open a file browser window. Navigate to the Include folder of the OSTRICH repository. Select all files in the folder and click "Add". If done correctly, the OSTRICH header files will now appear within the Header Files folder of Visual Studio. To add the source code, right click on the "Source Files" folder of the Solution Explorer and choose "Add->Existing Item". This time navigate to the OSTRICH src folder, select all of its contents, and click "Add". The OSTRICH source files should now appear in the Source Files folder.

2.3 Adding dependencies into Visual Studio

The external libraries OSTRICH uses also need to be added. These steps will add the libaries into Visual Studio.

2.3.1 Microsoft MPI

Right click on Resource Files within the Solution Explorer and select "Add->Existing Item". If you've installed the MPI SDK into the default location, you'll be able to find it within C:\Program Files (x86)\Microsoft SDKs\MPI\Lib\x64 on your machine. Select msmpi.lib and click "Add". If you installed the MPI SDK to a custom location, browse to that location and select the mpi library.

2.3.2 Microsoft ADO

Right click on Resource Files within the Solution Exporer and select "Add->Existing Item". If ADO is installed into the default location, you'll be able to find it within C:\Program Files\Common Files\System\ado on your machine. Select msado15.dll and click "Add".

2.4 Modifying compliation options

The next step is to modify the compilation options. Within the Solution Explorer, right click on your project name to bring up the Property Pages. Open the "Configuration Properties->C/C++->General" page. Within the Additional Include Directories box, add the following: C:\Program Files\Common Files\System\ado;C:\Program Files x86\Microsoft SDKs\MPI\Include;F:\ostrich\ostrich\include; The first two target the ADO and MPI headers, respectively. The last path points to the folder containing the header files within the OSTRICH repository. You will need to modify the final path as appropriate to target the directory on your system.

OSTRICH is setup to use the C++ 2017 standard. By default, Visual Studio utilizes an older 2014 C++ standard and must be told to build against the newer standard. Within the "Configuration Properties->General" page, there is the option to set the "C++ Language Standard". Update this to be the "ISO C++ 17 Standard"

We also need to disable compilation warning flags. The Microsoft compiler stack defaults to different input/output functions compared to Linux compiler stacks that aren't default in the C++ standards. To force the Microsoft compiler to use the default functions, we need to add a flag into the preprocessor options. With the Property Pages, open "Configuration Properties->C/C++->Preprocessor". Within the "Preprocessor Definitions" box, add _CRT_SECURE_NO_WARNINGS. This will tell the compiler to default to the C++ standard. Additionally, there is an known issue with how Visual Studio handles the C++ 2017 byte type. To force Visual Studio to compile correctly against the C++ 2017 standard, the "_HAS_STD_BYTE=0" option must also be added to the "Preprocessor Definitions" box.

2.5 Troubleshooting

If you're seeing header missing issues, double check that the configuration you set the compilation options for is the same as the one that is enabled in the Visual Studio IDE. In general, it's recommended to use x64. Confirm that you set the compilation options for the architecture/release combination you're using within the IDE.