Getting Started

This document attempts to explain the OpenMDAO development process and how a developer should interact with the various tools the project uses for version control, testing, deployment, etc. The source files for the Developer Guide can be found in the docs/dev-guide directory in the top level of your OpenMDAO source repository.

System Requirements

Working with OpenMDAO as a developer means there are some additional system requirements besides those mentioned in the User Guide. Make sure that you have installed all of those requirements first, and then add the other requirements described below.

Git

We use Git for version control. You’ll need it to access the OpenMDAO source repository. GitHub, where our source repository is stored, has excellent documentation describing how to install Git and how to become familiar with Git and GitHub. You can find it here.

C/C++ and Fortran Compilers

Certain packages used in OpenMDAO contain Python extensions, meaning that they contain non-Python source code that must be compiled. Packages currently in use require either C/C++ or Fortran compilers.

  • Linux:

    • gcc
    • gfortran

    If these compilers are not already on your system, they should be easily installable using your package manager. OpenMDAO currently builds and passes all tests with gcc/gfortran 4.1.2. We expect that later versions of gcc/gfortran 4.X should also work.

  • Mac OS X:

    • Xcode

      You can get the latest version from Apple, but to do so you’ll have to register (free) as a developer or log in if you are already registered. If you’re using Lion or Mountain Lion, you can also get Xcode from the App store. If you don’t want to get the whole X-Code environment, for Lion and Mountain Lion Apple offers a smaller compiler-only library called Command Line Tools for Xcode. You can get the latest Xcode or the smaller compiler package here.

    • gfortran

      It’s sometimes hard to figure out which version of gfortran to install on your Mac. See this page for a pretty good overview of what’s available. Alternatively, you can follow the instructions for installing a package manager called Homebrew, which will figure out the right kind of gfortran for you.

    • Lion (OS X 10.7) and Mountain Lion (OS X 10.8)

      • If you want to get the full Xcode, make sure version 4.3.2 or higher is installed. You can try the Mac App Store. When you install Xcode (via the app store or not), it does not install the compilers you need by default. So go to Xcode’s Preferences menu, choose Downloads, and then choose Command Line Tools.

        Screenshot of Xcode's Downloads screen showing options

        XCode’s Downloads Screen

      • gfortran 4.6.2, or use Homebrew

    • On Leopard:

      • gfortran - Click on fortran-macosx-leopard-x86.dmg under Miscellaneous Downloads.

        If you have g77 installed on Leopard, you may get build errors like: ld: library not found for -lcc_dynamic. This indicates that g77, which won’t work, is being used instead of gfortran. At the moment, the recommended fix is to change the name of g77 to something else, for example, _g77 so that it won’t be found by numpy.distutils.

  • Windows:

    • mingw32 (for Fortran and C++)

      You can find mingw32 here. You must do the following things when installing it:

      • Check the C++ and Fortran compiler installation options to get g++ (required to run OpenMDAO)

      • Create a file in your home directory called pydistutils.cfg that contains the following lines:

        [build_ext]
        compiler=mingw32
        

        On Windows, your home directory can be determined by typing the following command in your command-prompt:

        echo %USERPROFILE%
      • Make sure to put the bin directory of the mingw32 install in your path.

        To check if this directory is already in your path, type the following in your command-prompt:

        echo %PATH%

        If you do not see the directory, you can add it to your path once again from command-prompt:

        set PATH=%PATH%;/path/to/bin_dir
    • Visual C++ 2008 (Optional)

      You can optionally use Visual C++ 2008 as your C++ compiler. You don’t need it; mingw32 will work fine, but if you prefer Visual C++ 2008, you’re welcome to use it instead. The Express version will work, but others (Professional, Standard) should work too. To get this software, go to the downloads page.

System Configuration

Some steps of the development process, e.g., downloading a branch of the source repository and downloading Python distributions, require network access. If you’re using Linux or Mac OS X and are behind an http proxy, you may have to set the http_proxy environment variable on your system for Git and virtualenv to function properly. If you’re using Windows 7, please follow this link for information on configuring proxy settings.

Using Git and GitHub

The source repository for the OpenMDAO project is available on GitHub. There is a wealth of good documentation available online about Git and Github itself. The GitHub help page is a good place to start. If you’re a Windows user, make sure to read the details about using Git Bash on the Windows specific installation instructions. The Pro Git book is also excellent. It’s very important to take a look at these, because Git differs from other version control systems in some significant ways.

The first major difference is that Git has a staging area that files must be placed in before they’re committed. Luckily the git commit command has an option, -a, that will eliminate this odd behavior and commit all of the modified files in the repository without having to stage them first. See the Committing Changes section for further explanation of git commit.

The other major difference is how branches are handled. In Git, creating a branch does not create a separate copy of the repository, but instead is basically a pointer to a commit history within the repository. This makes Git branches cheap to create. This means that you should not hesitate to make a new branch when working on something. This will be discussed a little more below in the Getting the Source Code section.

The following figure gives an overall view of the version control process while using Git and GitHub.

GitHub version control process with arrows showing relationship among OpenMDAO repository, personal fork of that repo, and personal copy of OpenMDAO repo with its branches

Version Control Process

From the OpenMDAO repository on GitHub, you create a personal fork of the project, so there is now a repository that you can write to. In addition to creating a fork, you must also create a clone of the OpenMDAO repository on your machine. This is the repository that you will be working directly off of, via the creation of branches. Once you are satisfied with a branch’s development, you can push this branch back up to your personal fork. From here, if you would like to contribute this work back to the OpenMDAO repository, you must issue a pull request to have your modifications merged in.

Git User Setup

If you have not previously used Git on a particular machine where you intend to work with Git repositories, you should follow the instructions here to set your username, email, and API token. This way, your contact information will be included whenever you commit to a repository on that machine.

If you’re using Git on Windows, to follow the above instructions you’ll first need to create a bash terminal by running Git -> Git Bash from the Windows start menu.

Accessing OpenMDAO on GitHub – SSH Keys

The source repository for OpenMDAO is read-accessible to the public, so making a clone of it does not require that you have a GitHub account. If you plan to contribute to the OpenMDAO project on GitHub, you will need to have a GitHub account and to register your public SSH key with that account. The following section describes how to register your SSH key with GitHub.

These instructions assume that you already have a GitHub account. If you do not, please go to https://github.com and register for an account. Go ahead and log in to your GitHub account, since you will need to be logged in to register your key.

If you’re using Git on Windows, to follow the instructions below you’ll first need to create a bash terminal by running Git --> Git Bash from the Windows start menu.

Creating Your Key

  1. First, check to see if you already have an SSH key. Look for a file called ~/.ssh/id_rsa.pub. If the file is there, skip to the next section and learn how to register your key with GitHub.
  2. You should be in your home directory on your Linux machine. At the prompt, type: ssh-keygen -t rsa -C "your_email@youremail.com".
  3. When prompted, press Enter to accept the default file name for your key.
  4. Press Enter when prompted for a password and then press it again to confirm that you are NOT entering a password. Your key pair is stored in ~/.ssh/ as id_rsa.pub (public key) and id_rsa (private key).

Note

In the unusual event that the ssh-keygen command fails, you may need to install OpenSSH. To do this requires that you have admin privileges. On Ubuntu, you can install OpenSSH by opening your terminal and typing: sudo apt-get install openssh-client.

Registering the Key with GitHub

You need to register and upload the public portion of your SSH key to GitHub.

  1. Open your public key in a text editor and copy its contents to your clipboard. The public key file has the extension .pub; for example: id_rsa.pub
  2. You must be logged into GitHub for this step. Go to Account Settings –> SSH Public Keys, then click on Add another public key.
  3. Enter a name into the Title text field to remind you which machine/account you’re entering ssh information for.
  4. Paste your public key into the Key text box and then click the Add Key button (below the text box) to continue.

Getting the Source Code

The official OpenMDAO-Framework repository lives on GitHub at https://github.com/OpenMDAO/OpenMDAO-Framework.

To create a local OpenMDAO-Framework repository, you need to clone the OpenMDAO-Framework repository on GitHub using the following command:

git clone git://github.com/OpenMDAO/OpenMDAO-Framework.git

or, if the port that git:// uses is blocked by your firewall, try this:

git clone http://github.com/OpenMDAO/OpenMDAO-Framework.git

Making a Personal Fork of OpenMDAO-Framework

If you intend to make contributions to the project, you’ll need to make your own personal fork of OpenMDAO-Framework on GitHub. Making your own fork is easy; just log into GitHub, go to the OpenMDAO-Framework repository page at https://github.com/OpenMDAO/OpenMDAO-Framework, and click the Fork button near the top of the page.

Later, when you finish working on a branch in your local repository, you’ll be able to push it up to your personal fork and issue a pull request to get your changes into the dev branch of the official repository.

Layout of a Source Repository

The directory structure of your repository should look like this:

contrib
The directory containing source to be packaged into distributions that can be released separately from OpenMDAO. These distributions may or may not depend upon OpenMDAO. Distributions that have not yet been approved to be part of openmdao.lib can live here – as long as their license is compatible with Apache V2.0. No proprietary code or GPL code should be placed in the OpenMDAO-Framework repository.
devenv
The directory containing the OpenMDAO virtual environment. Note that this is not part of the source repository. You will build it by running the go-openmdao-dev.py script that sits at the top of the source repository. See Creating the Virtual Environment.
docs
The directory containing all user documentation for OpenMDAO. The documentation is broken up into several major documents, each found in a separate subdirectory, e.g., plugin-guide contains the Plugin Developer Guide, dev-guide contains the Developer Guide, and so on.
examples
Python package containing examples of using OpenMDAO.
misc
The directory containing miscellaneous scripts and configuration files used by OpenMDAO developers.
openmdao.devtools
Python package containing scripts intended for developers and maintainers of openmdao to do things like build the docs or create a release. These scripts assume that the source repository is present, so this package is not distributed as part of an OpenMDAO release.
openmdao.lib
Python package containing source for the OpenMDAO standard library of modules.
openmdao.main
Python package containing all infrastructure source for OpenMDAO.
openmdao.test
Python package containing source for various OpenMDAO plugins used for testing.
openmdao.units
Python package containing tools for doing unit conversion.
openmdao.util
Python package containing source for various Python utility routines used by OpenMDAO developers.

Layout of a Namespace Package

OpenMDAO is split up into multiple Python packages, all under a top level package called openmdao. This top package, called a namespace package, is a sort of fake package that allows us to maintain and release our subpackages separately while appearing to the user to be all part of the same top level package. The following packages under the openmdao namespace have a similar directory layout: openmdao.main, openmdao.lib, openmdao.devtools, openmdao.util and openmdao.test. The layout is shown below.

openmdao.<package>
The top level directory for the package denoted by <package>. This contains the setup.py script which is used to build and create a distribution for the package.
openmdao.<package>/src
Contains all of the package source code.
openmdao.<package>/src/openmdao
Contains a special __init__.py file and a <package> subdirectory.
openmdao.<package>/src/openmdao/<package>
Contains the actual source code, usually a bunch of Python files. There could be a standard Python package directory structure under this directory as well.
openmdao.<package>/src/openmdao/<package>/test
Contains unit tests for this package. These are executed by openmdao test.