2. Loading Data

In a visualization pipeline, data sources bring data into the system for processing and visualization. Sources, such as the Sphere source (accessible from the Sources menu in paraview), programmatically create datasets for processing. Another type of data sources are readers. Readers can read data written out in disk files or other databases and bring it into ParaView for processing. ParaView includes readers that can read several of the commonly used scientific data formats. It’s also possible to write plugins that add support for new or proprietary file formats.

ParaView provides several sample datasets for you to get started. You can download an archive with several types of data files from the download page at https://www.paraview.org/download under the Data section.

2.1. Opening data files in paraview

To open a data file in paraview, you use the Open File dialog. This dialog can be accessed from the File > Open menu or by using the button in the Main Controls toolbar. You can also use the keyboard shortcut CTRL + O (or ⌘ + O) to open this dialog.

Fig. 2.1 Open File dialog in paraview for opening data (and other) files.

The Open File dialog allows you to browse the file system on the data processing nodes. This will become clear when we look at using ParaView for remote visualization. While several of the UI elements in this dialog are obvious such as navigating up the current directory, creating a new directory, and navigating back and forth between directories, which can all be done with the standard system shortcuts like CTRL + N (or ⌘ + N) to create a directory or Alt + ↑ to go to the parent directory, there are a few things to note.

• The Favorites pane contains a custom list of favorite directories that can be customized using the buttons above it, which respectively adds the current directory to the favorites or removes all the favorites. Another way to change which directories are in the favorites is to use the right-click context menus in the list of files. Right-clicking any directory in the main list will display a menu with the option to add it to the favorites. To remove edit an exising favorite, right-click or ctrl-click the favorite. A context menu appears which lets you remove the favorite or rename it.

• The Locations pane shows some platform-specific common locations such as the home directory and desktop. External drives will also appear in this list.

• The Recent Directories pane shows a few of the most recently used directories.

You can browse to the directory containing your datasets and either select the file and hit Ok or simply double click on the file to open it. You can also select multiple files using the CTRL (or ⌘) key. This will open each of the selected files separately.

Right-clicking the files list will display a few options, depending on what was right-clicked.

• Selecting a directory adds the Add to favorites option which adds the selected directory to the Favorites pane on the left.

• Selecting a file or a directory adds the Rename option.

• The Open in file explorer is also always present and either opens in the system file explorer the selected directory if one was right-clicked, or opens the current directory if a file or nothing was selected.

• Selecting an empty directory provides the Delete empty directory option.

• The Show Hidden Files is always visible, and can be checked to display the hidden files and directories.

The Look in: combobox shows the current path in the Open File dialog. Clicking on the combobox down arrow will show the parent directories of the current directory while clicking in the path line edit itself allows you to edit the path. Navigation buttons showing arrows are to the right of the combo box. These enable navigating through previously visited directories as well as moving up to the parent of the current directory. The button to the right of the arrows toggles on or off additional columns in the dialog that show file size and last modified date.

When a file is opened, paraview will create a reader instance of the type suitable for the selected file based on its extension. The reader will simply be another pipeline module, similar to the source we created in Section 1. From this point forward, the workflow will be the same as we discussed in Section 1.4.2 : You adjust the reader properties, if needed, and hit Apply . paraview will then read the data from the file and render it in the view.

If you selected multiple files using the CTRL (or ⌘) key, paraview will create multiple reader modules. When you hit Apply , all of the readers will be executed, and their data will be shown in the view.

Fig. 2.2 The Open File dialog can be used to select a temporal file series (top) or select multiple files to open separately (bottom).

By default, paraview groups files that appear to define a time series. These file series have names that contain a number sequence where the number defines the order of the files in time. paraview can be told not to do this by clicking on the button at the top right of the Open File dialog.

Did you know?

This ability to hit the Apply button once to accept changes on multiple readers applies to other pipeline modules, including sources and filters. In general, you can change properties for multiple modules in a pipeline, and hit Apply to accept all of the changes at once. It is possible to override this behavior from ParaView’s’ Settings dialog using the Auto Apply Active Only setting under the General tab. When this setting is enabled, only the selected source in the Pipeline Browser will be updated.

2.1.1. Reader selection depending on the selected file types

Fig. 2.3 File dialog “Files of type” to filter which files to show and to change how the reader will be chosen.

When selecting files, the current category for the “Files of type” field (Fig. 2.3) changes the way the reader will be chosen.

• If the “Supported Files” value is selected, if only one reader is available for the type of file, it will be selected automatically. If multiple readers can be used, the “Open Data With…” dialog (Fig. 2.4) will appear to choose which reader to use. Clicking the “Set reader as default” button tells ParaView to automatically use this reader when one of the file name patterns it says it can support matches a file that could otherwise be opened by multiple readers. When a reader is set as default, it will be used automatically for files that match its patterns.

Fig. 2.4 Open Data With... dialog shown to manually choose the reader to use for a file with multiple available readers.

• If the “All Files” value is selected, the same dialog will be displayed with all the existing readers. If you picked an incorrect reader, however, you’ll get error messages either when the reader module is instantiated or after you hit Apply. In either case, you can simply Delete the reader module and try opening the file again, this time choosing a different reader. If you can click the Set reader as default button, a small window with a line edit will be displayed where you can set the custom pattern to use for this reader (Fig. 2.5). This is a standard wildcard pattern, and multiple patterns can be used by separating them with spaces.

Fig. 2.5 The Default reader details setting is used to define file name patterns and which reader should read files that match those patterns.

• If a specific reader is selected, for example “PNG Image Files”, this reader will be always be used automatically, even if other readers would be available or if the file is matching a pattern present in the “Default reader details” settings.

Error messages in paraview are shown in the Output Messages window (Fig. 2.6). It is accessible from the View > Output Messages menu. Whenever there’s a new error message, paraview will automatically pop open this window and raise it to the top (as long as the Always open for new messages option is set. This window can be attached, or docked, in the main window so that it is visible with the other user interface elements without covering them up.

Fig. 2.6 The Output Messages window is used to show errors, warnings, and other messages raised by the application.

The default readers settings can be seen and modified in the Edit > Settings menu, and IO tab (Fig. 2.7).

Fig. 2.7 Default reader settings.

2.1.2. Handling temporal file series

Most datasets produced by scientific simulation runs are temporal in nature. File formats differ in how this information is saved in the file. While several file formats support saving multiple timesteps in the same file, others save them out as a sequence of files, known as a file series

The Open File dialog automatically detects file series and shows them as a grouped element, as shown in Fig. 2.2. To load the file series, simply select the group, and hit Ok . You can also open a single file in the series as a regular file. To do so, open the file group and select the file you want to open.

paraview automatically detects several of the commonly-known file naming patterns used for indicating a file series. These include:

fooN.vtk	fooN.vtk	Nfoo.vtk	foo.vtk.N
foo_N.vtk	foo.N.vtk	N.foo.vtk	foo.vtksN

where foo could be any filename, N is a numeral sequence (with any number of leading zeros), and vtk could be any extension.

Sometimes this grouping of files with names that follow a numeral sequence into a time series is not desired. The Open File dialog has a toggle button that controls whether these file sequences are interpreted as time series. It defaults to on, but if toggled off, then paraview will not group file series until the button is toggled back on.

ParaView also supports a meta file format based on JSON. This format has support for specifying time values. The format (currently version 1.0) looks like the following

{
  "file-series-version" : "1.0",
  "files" : [
    { "name" : "foo1.vtk", "time" : 0 },
    { "name" : "foo2.vtk", "time" : 5.5 },
    { "name" : "foo3.vtk", "time" : 11.2 }
  ]
}

Usually, a reader supporting extension will also supports extension.series as the file series meta file.

2.1.3. Dealing with time

When you open a dataset with time, either as a file series or in a file format that natively supports time, paraview will automatically set up an animation for you so that you can play through each of the time steps in the dataset by using the button on the VCR Controls toolbar (Fig. 2.8). You can change or modify this animation and further customize it, as discussed in Chapter Section 7.

Fig. 2.8 VCR Controls toolbar for interacting with an animation.

2.1.4. Reopening previously opened files

paraview remembers most recently opened files (or file series). Simply use the File > Recent Files menu. paraview also remembers the reader type selected for files with unknown extensions or for occasions when multiple reader choices were available. Lastly, ParaView also records the server connection used to load a particular file. If you are connected to one server and access a file through the Recent Files menu that was on another server, paraview will ask if you want to disconnect from the current server and connect to the server on which the recent file resides.

2.1.5. Opening files using command line options

paraview provides a command line option that can be used to open datasets on startup.

> paraview --data=.../ParaViewData/Data/can.ex2

This is equivalent to opening a can.ex2 data file from the Open File dialog. The same set of follow-up actions happen. For example, paraview will try to locate a reader for the file, create that reader, and wait for you to hit Apply .

To open a file series, simply replace the numbers in the file name sequence by a . For example, to open a file series named my0.vtk, my1.vtk … myN.vtk, use my..vtk.

> paraview --data=.../ParaViewData/Data/my..vtk

2.1.6. Common properties for readers

ParaView uses different reader implementations for different file formats. Each of these have different properties available to you to customize how the data is read and can vary greatly depending on the capabilities of the file format itself or the particular reader implementation. Let’s look at some of the properties commonly available in readers.

2.1.6.1. Selecting data arrays

Fig. 2.9 Array selection widget for selecting array to load from a data file.

One of the most common properties on readers is one that allows you to select the data arrays to be loaded, be they cell-centered, point-centered, or otherwise. Often times, loading only the data arrays you know you are going to use in the visualization will save memory, as well as processing time, since the reader does not have to read in those data arrays, and the filters in the pipeline do not have to process them.

Did you know?

You can change paraview’s default behavior to load all available data arrays by selecting the Load All Variables checkbox under Settings/Properties Panel Options/Advanced .

The user interface for selecting the arrays to load is simply a list with the names of the arrays and a checkbox indicating whether that array is to be loaded or not (Fig. 2.9). Icons, such as and are often used in this widget to give you an indication of whether the array is cell-centered or point-centered, respectively.

If you initially de-select an array, but then as you’re setting up your visualization pipeline realize that you need that data array, you can always go back to the Properties page for the reader by making the reader active in the Pipeline Browser and then changing the array selection. ParaView will automatically re-execute any processing pipeline set up on the reader with this new data array.

Common Errors

Remember to hit Apply (or use Auto Apply ) after changing the array selection for the change to take effect.

Sometimes the list of data arrays can get quite large, and it can become cumbersome to find the array for which you are looking. To help with such situations, paraview provides a mechanism to search lists. Click inside the widget to make it get the focus. Then type CTRL + F (or ⌘ + F) to get a search widget. Now you can type in the text to search. Matching rows will be highlighted (Fig. 2.10).

Fig. 2.10 To search through large lists in paraview, you can use CTRL + F.

Did you know?

The ability to search for items in an array selection widget also applies to other list and tree widgets in the paraview UI. Whenever you see a widget with a large number of entries in a list, table, or tree fashion, try using CTRL + F (or ⌘ + F).

2.2. Opening data files in pvpython

To open data files using the scripting interface, ParaView provides the OpenDataFile function.

>>> reader = OpenDataFile(".../ParaViewData/Data/can.ex2")
>>> if reader:
...   print("Success")
... else:
...   print("Failed")
...

OpenDataFile will try to determine an appropriate reader based on the file extension, just like paraview. If no reader is determined, None is returned. If multiple readers can open the file, however, OpenDataFile simply picks the first reader. If you explicitly want to create a specific reader, you can always create the reader by its name, similar to other sources and filters.

>>> reader = ExodusIIReader(FileName=".../ParaViewData/Data/can.ex2")

To find out information about the reader created and the properties available on it, you can use the help function.

>>> reader = ExodusIIReader(FileName=".../ParaViewData/Data/can.ex2")
>>> help(reader)
Help on ExodusIIReader in module paraview.servermanager object:

class ExodusIIReader(ExodusIIReaderProxy)
 |  The Exodus reader loads
 |  Exodus II files and produces an unstructured grid output.
 |  The default file extensions are .g, .e, .ex2, .ex2v2,
 |  .exo, .gen, .exoII, .exii, .0, .00, .000, and .0000. The
 |  file format is described fully at:
 |  http://endo.sandia.gov/SEACAS/Documentation/exodusII.pdf.
 |  ...
 |
 |  -----------------------------------------------------------------
 |  Data descriptors defined here:
 |
 |  AnimateVibrations
 |      If this flag is on and HasModeShapes is also on, then
 |      this reader will report a continuous time range [0,1] and
 |      animate the displacements in a periodic sinusoid. If this
 |      flag is off and HasModeShapes is on, this reader ignores
 |      time. This flag has no effect if HasModeShapes is off.
 |
 |  ApplyDisplacements
 |      Geometric locations can include displacements. When this
 |      option is on, the nodal positions are 'displaced' by the
 |      standard exodus displacement vector. If displacements are
 |      turned 'off', the user can explicitly add them by applying
 |      a warp filter.
 |  ...

Did you know?

The help function can be used to get information about properties available on any source or filter instance. It not only lists the properties, but also provides information about how they affect the pipeline module. help can also be used on functions. For example:

>>> help(OpenDataFile)

Help on function OpenDataFile in module paraview.simple:

OpenDataFile(filename, \*\*extraArgs)
    Creates a reader to read the given file, if possible.
    This uses extension matching to determine the best reader
    possible. If a reader cannot be identified, then this
    returns None.

2.2.1. Handling temporal file series

Unlike paraview, pvpython does not automatically detect and load file series. There are two ways you can load a file series:

• You can explicitly list the filenames in the series and pass that to the OpenDataFile call.

# Create a list with the names of all the files in the file series in
# correct order.
>>> files = [".../Data/multicomb_0.vts",
             ".../Data/multicomb_1.vts",
             ".../Data/multicomb_2.vts"]
>>> reader = OpenDataFile(files)

• You can use globbing by inserting the wildcard * in the file name using the utility paraview.util.Glob. This utility runs fnmatch python package on the server’s file system, so any pattern supported by fnmatch is supported and interpreted by this utility.

>>> import paraview.util
# Create a list of names of all the files in the file series.
>>> files = paraview.util.Glob(path = "multicomb_*.vts", rootDir = ".../Data")
>>> reader = OpenDataFile(files)

2.2.2. Dealing with time

Similar to paraview, if you open a time series or a file with multiple timesteps, pvpython will automatically set up an animation for you to play through the timesteps.

>>> files = [".../Data/multicomb_0.vts",
             ".../Data/multicomb_1.vts",
             ".../Data/multicomb_2.vts"]
>>> reader = OpenDataFile(files)
>>> Show()
>>> Render()

# Get access to the animation scene.
>>> scene = GetAnimationScene()
# Now you use the API on the scene when doing things such as playing
# the animation, stepping through it, etc.

# This will simply play through the animation once and stop. Watch
# the rendered view after you hit `Enter.'
>>> scene.Play()

2.2.3. Common properties on readers

2.2.3.1. Selecting data arrays

For those properties on readers that allow you to control what to read in from the file such as point data arrays, cell data arrays, or data blocks, paraview uses a selection widget, as seen in Section 2.1.6.1. Likewise, pvpython provides an API that allows you to determine the available options and then select/deselect them.

The name of the property that allows you to make such selections depends on the reader itself. When in doubt, use the tracing capabilities in paraview (Section 1.6.2) to figure it out. You can also use help (Section 2.2).

ExodusIIReader has a PointVariables property that can be used to select the point data arrays to load. Let’s use this as an example.

# Open an ExodusII data file.
>>> reader = OpenDataFile(".../Data/can.ex2")

# Alternatively, you can explicitly create the reader instance as:
>>> reader = ExodusIIReader(FileName = ".../Data/can.ex2")

# To query/print the current status for `PointVariables' property,
# we do what we would have done for any other property:
>>> print(GetProperty("PointVariables"))
['DISPL', 'VEL', 'ACCL']

# An alternative way of doing the same is as follows:
>>> print(reader.PointVariables)
['DISPL', 'VEL', 'ACCL']

# To set the property, simply set it to list containing the names to
# enable, e.g., if we want to read only the 'DISPL' array, we do
# the following:
>>> SetProperties(PointVariables=['DISPL'])

# Or using the alternative way for doing the same:
>>> reader.PointVariables = ['DISPL']

# Now, the new value for PointVariables is:
>>> print(reader.PointVariables)
['DISPL']

# To determine the array available, use:
>>> print(reader.PointVariables.Available)
['DISPL', 'VEL', 'ACCL']
# These are the arrays available in the file.

Changing PointVariables only changes the value on the property. The reader does not re-execute until a re-execution is requested either by calling Render or by explicitly updating the pipeline using UpdatePipeline .

>>> reader.PointVariables = ['DISPL', 'VEL', 'ACCL']

# Assuming that the reader is indeed the active source, let's update
# the pipeline:
>>> UpdatePipeline()

# Or you can use the following form if you're unsure of the active
# source or just do not want to worry about it.
>>> UpdatePipeline(proxy=reader)

# Print the list of point arrays read in.
>>> print(reader.PointData[:])
[Array: ACCL, Array: DISPL, Array: GlobalNodeId, Array: PedigreeNodeId, Array: VEL]

# Change the selection.
>>> reader.PointVariables = ['DISPL']

# Print the list of point arrays read in, nothing changes!
>>> print(reader.PointData[:])
[Array: ACCL, Array: DISPL, Array: GlobalNodeId, Array: PedigreeNodeId, Array: VEL]

# Update the pipeline.
>>> UpdatePipeline()

# Now the arrays read in has indeed changed as we expected.
>>> print(reader.PointData[:])
[Array: DISPL, Array: GlobalNodeId, Array: PedigreeNodeId]

We will cover the reader.PointData API in more details in Section 3.3.

2.3. Reloading files

While ParaView is often used after the simulation has generated all the data, it is not uncommon to use ParaView to inspect data files as they are being written out by the simulation. In such cases, the simulation may either be modifying existing file(s) with new timesteps or creating new files for each timestep. In such cases, you may want to refresh ParaView to make it aware of the changes. In paraview, this can be done using Reload Files . When the reader is active, you can use the File > Reload Files menu to request the reader to refresh. paraview will prompt you to choose whether to reload the existing file(s) or look for new files in the file series, as shown in Fig. 2.11. Click on Reload existing file(s) , to force the reader to re-read the files already opened. This is useful in cases where the simulation may have modified existing file(s). Use Find new files to make the reader aware of any new files in the file series.

Fig. 2.11 The Reload Options dialog allows you to choose how to refresh the reader.

Similar to paraview, in pvpython, you use ReloadFiles to reload existing files, and ExtendFilesSeries to look for new files in a file series.

# For file being modified in place per timestep
>>> reader = OpenDataFile(file)
...
>>> ReloadFiles(reader)

# For files being generated per timestep
>>> reader = OpenDataFile(file)
...
>>> ExtendFilesSeries(reader)