With the dispatch of the AuroraWatch schools' magnetometers imminent I have implemented a
Python toolkit to process the data. The
numpy and
matplotlib modules are used extensively. The toolkit provides an
API to load data and perform various processing actions on it, such as plotting data. The concept is influenced by my previous
Multi-instrument Analysis toolbox for Matlab. In addition to the loading and processing of magnetic field data
auroraplot allows other data types to be added later.
Loading data
Data can be loaded with arbitrary start and end times very simply:
md = ap.load_data('AURORAWATCHNET', 'LAN1', 'MagData',
np.datetime64('2013-09-20T12:00:00+0000'),
np.datetime64('2013-09-21T12:00:00+0000'))
In this case the selected portion of data crosses midnight and two data files must be loaded, concatenated and trimmed to get the desired time range. This is performed automatically by
auroraplot, the user need not be concerned with the format of the files or where they are located. It is even possible for the files to be downloaded on-the-fly using FTP or HTTP transfer protocols.
load_data returns an object (of type
MagData) to the user containing the actual magnetic field data and various other metadata, such as a timestamp for each sample and the data units. Each object can store more multiple data
channels but all data points must share the same timestamps, be of the same type and share the same units. Therefore it is not possible to store operating temperatures (units °C) in an object holding magnetic field strength (units tesla). The operating temperature data can be accessed as:
td = ap.load_data('AURORAWATCHNET', 'LAN1', 'TemperatureData',
np.datetime64('2013-09-20T12:00:00+0000'),
np.datetime64('2013-09-21T12:00:00+0000'))
Battery voltage (data type
VoltageData) can be accessed in a similar same way.
Plotting data
High-level plot functions enable the data be be plotted very simply, for the magnetic field data loaded previously
md.plot()
will produce a matplotlib figure with a title and the axes labelled with the correct units. Temperature and voltage data are plotted in the same way.
I have created some tools to make working with numpy's datetime64 and timedelta64 objects more convenient, including rounding functions (
round,
ceil and
floor) which round to an interval. They are useful for finding the start of an hour, or the end of a day. I have also created
Locator and
Formatter classes to sensibly label time axes using
datetime64 times and
timedelta64 intervals. Tick marks are located on the nearest second, minute, hour, day, month or year boundary (or multiple thereof) depending on the time interval being displayed. Thanks to matplotlib's structure the labels are automatically regenerated with the most appropriate time units when a plot is zoomed.
Other operations
Quiet-day curves
Other operations include the generation of
quiet-day curves. These are the curves from which we measure geomagnetic activity and are of critical importance for AuroraWatch UK. There are are not flat but have a daily variation caused by the
equatorial electrojet. The empirical algorithm selects the days (typically 5) with the least geomagnetic activity. A truncated Fourier series is used to guarantee that the quiet-day curves are cyclic, with the start and end points having the same magnitude and slope. This is essential otherwise our rolling plots would show up the discontinuities in the QDC at midnight, and would falsely cause step changes in the geomagnetic activity. An example QDC is shown below.
 |
Quiet-day curve for magnetometer at Lancaster ,UK. This is derived from recorded data
and clearly shows the Sq current system caused by the equatorial electrojet. |
From this we can see that even on a geomagnetically quiet day we would expect a 30nT variation in field strength seen by the magnetometer. The AuroraWatch threshold for minor geomagnetic activity is 50nT so this shows the importance of using a quiet-day curve instead of a flat line when calculating geomagnetic activity.
Stack plots
Stack plots (also called
magnetograms) are a convenient representation for magnetic field data from a set of magnetometers separated in latitude.Data from the northernmost instruments is placed at the top and that from the southernmost at the bottom. An example stackplot is shown below:
 |
| Stackplot showing data from two Lancaster stations and from Ormskirk. |
The magnetometer at Ormskirk is operated by the Met Office as part of a test. The stackplots will be more interesting as the network grows.
Open source
The source code is available under a BSD-type license from
Github.You will need python, along with the numpy (version 1.7), matplotlib and scipy python modules.auroraplot has been tested under Debian Linux (64 bit version) and Raspbian on the Raspberry Pi.
