tag:blogger.com,1999:blog-2526424532955904532024-02-20T00:03:01.055+00:00Steve Marple's blogSteve Marplehttp://www.blogger.com/profile/15582257560382364498noreply@blogger.comBlogger43125tag:blogger.com,1999:blog-252642453295590453.post-62723575705492497952018-02-07T22:28:00.000+00:002018-02-07T22:28:13.104+00:00Python one-liner: return latest version<div dir="ltr" style="text-align: left;" trbidi="on">Today I need to find the latest version of some software. A normal ASCII sort (or even a <a href="http://man7.org/linux/man-pages/man1/sort.1.html#DESCRIPTION" rel="nofollow" target="_blank">numeric one</a>), would be unreliable given the use of <a href="https://semver.org/" rel="nofollow" target="_blank">semantic version numbers</a>. Instead I came up with this simple Python command that uses distutils <code>LooseVersion</code> to sort in the correct order:<br />
<br />
<blockquote class="tr_bq"><code>python -c "import sys;from distutils.version import LooseVersion;print(sorted(sys.stdin.read().split(),key=LooseVersion)[-1])"</code></blockquote><br />
Simply pipe the input into <code>stdin</code> and Python will emit the latest version from the list of words or lines. For example:<br />
<blockquote class="tr_bq"><code>$ echo octocat-0.0.2 octocat-10.0.3 octocat-1.2.0a octocat-6.1.0alpha | python -c "import sys;from distutils.version import LooseVersion;print(sorted(sys.stdin.read().split(),key=LooseVersion)[-1])"</code></blockquote><div>returns</div><blockquote class="tr_bq"><code>octocat-10.0.3</code></blockquote><div><br />
</div><br />
</div>Steve Marplehttp://www.blogger.com/profile/15582257560382364498noreply@blogger.com0tag:blogger.com,1999:blog-252642453295590453.post-22388417251598248172017-08-19T12:08:00.004+01:002017-08-27T12:13:21.860+01:00Using the Raspberry Pi sense hat to show the AuroraWatch UK status <div dir="ltr" style="text-align: left;" trbidi="on">
<table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto; text-align: center;"><tbody>
<tr><td style="text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjBd1BuEKg6z1UfnJccLhj6yyU5uZfaa4iGUwnaKsLnguRVm6cmtqm6h9SLhZJM510phRe3p7UPwHvzs5Y32tlAUeOgmNmf5N3qf_0RP6178NWjEXKKJP9u5FAvTTL28cj1_Qig64JeqCQ/s1600/awuk-status-rpi-sense-hat.jpg" imageanchor="1" style="margin-left: auto; margin-right: auto;"><img border="0" data-original-height="600" data-original-width="600" height="320" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjBd1BuEKg6z1UfnJccLhj6yyU5uZfaa4iGUwnaKsLnguRVm6cmtqm6h9SLhZJM510phRe3p7UPwHvzs5Y32tlAUeOgmNmf5N3qf_0RP6178NWjEXKKJP9u5FAvTTL28cj1_Qig64JeqCQ/s320/awuk-status-rpi-sense-hat.jpg" width="320" /></a></td></tr>
<tr><td class="tr-caption" style="text-align: center;">Image © Barry Osborne 2017. Used with permission.</td></tr>
</tbody></table>
<br />
<br />
This post uses the Raspberry Pi <a href="https://www.raspberrypi.org/products/sense-hat/" rel="nofollow" target="_blank">sense hat</a> to display the <a href="http://aurorawatch.lancs.ac.uk/" target="_blank">AuroraWatch UK</a> alert status. It assumes you already have Raspbian installed on a Raspberry Pi, networking configured and enabled, and the sense hat fitted to the Pi.<br />
<br />
The current status level is easily obtained from the AuroraWatch UK <a href="http://aurorawatch.lancs.ac.uk/api-info/" target="_blank">API</a> with a <a href="https://github.com/stevemarple/python-aurorawatchuk" rel="nofollow" target="_blank">custom python module</a>. First, ensure you have the necessary python modules installed from the Raspbian software archive:<br />
<br />
<span style="font-family: Courier New, Courier, monospace;">sudo apt-get update</span><br />
<span style="font-family: Courier New, Courier, monospace;">sudo apt-get install python-lxml python-requests python-six</span><br />
<br />
Then install two custom python modules. We will install them into the user profile, so run these commands as the user <span style="font-family: Courier New, Courier, monospace;">pi</span><span style="font-family: inherit;">:</span><br />
<br />
<span style="font-family: Courier New, Courier, monospace;">pip install --user --no-deps atomiccreate</span><br />
<span style="font-family: Courier New, Courier, monospace;">pip install --user --no-deps aurorawatchuk</span><br />
<br />
Finally, download the python script which fetches the status and displays it on the sense hat (I used <a href="https://bit.ly/" rel="nofollow" target="_blank">bit.ly</a> to shorten the original GitHub URL):<br />
<br />
<span style="font-family: Courier New, Courier, monospace;">wget https://bit.ly/awuk-status-on-sense-hat -O awuk_sense_hat.py</span><br />
<br />
The AuroraWatch UK status can then be displayed on the sense hat with this simple command:<br />
<br />
<span style="font-family: "courier new" , "courier" , monospace;">python</span><span style="font-family: "courier new" , "courier" , monospace;"> awuk_sense_hat.py</span><br />
<span style="font-family: "courier new" , "courier" , monospace;"><br />
</span><span style="font-family: inherit;">The program will monitor the current status and set the LED colours to show the status level (green, yellow, amber or red). For more information about the AuroraWatch UK status level meaning and interpretation see <a href="http://aurorawatch.lancs.ac.uk/alerts/" target="_blank">http://aurorawatch.lancs.ac.uk/alerts/</a>. If you don't have a sense hat (I don't) it is even possible to try this with the <a href="https://www.raspberrypi.org/blog/desktop-sense-hat-emulator/" target="_blank">sense hat emulator</a>.</span><br />
<h3 style="text-align: left;">
<span style="font-family: inherit;">Credits</span></h3>
<span style="font-family: inherit;">The geomagnetic activity status is courtesy of <a href="http://aurorawatch.lancs.ac.uk/" target="_blank">AuroraWatch UK</a> and uses data from the SAMNET and AuroraWatchNet magnetometers.See also the AuroraWatch UK <a href="http://aurorawatch.lancs.ac.uk/conditionsofuse/" target="_blank">conditions of use</a> for the API.</span><br />
<span style="font-family: inherit;"><br /></span>
<span style="font-family: inherit;">Thanks to Barry Osborne for testing the software with a real Raspberry Pi sense hat, and for providing the photograph.</span></div>
Steve Marplehttp://www.blogger.com/profile/15582257560382364498noreply@blogger.com1tag:blogger.com,1999:blog-252642453295590453.post-15050863420176235742017-02-19T12:17:00.001+00:002017-09-07T12:15:27.916+01:00AuroraWatch UK camera update #2<div dir="ltr" style="text-align: left;" trbidi="on">
<h3 style="text-align: left;">
A Python module for the AuroraWatch UK API</h3>
The first goal in the journey to create an automated all-sky camera system for <a href="http://aurorawatch.lancs.ac.uk/" target="_blank">AuroraWatch UK</a> was to interface to the camera using Python. If you missed update #1 you can read it at <a href="http://blog.stevemarple.co.uk/2016/12/aurorawatch-uk-camera-update-1.html" target="_blank">here</a>.<br />
<br />
One of the requirements of the camera software is to be able to change between different recording settings, for instance, in response to solar elevation and the AuroraWatch UK status level. Computing solar elevation is easily acheived with the <a href="https://pypi.python.org/pypi/astral/" target="_blank">astral</a> module. For the AuroraWatch UK status level I began with fetching the status XML document but soon realised a <i>much</i> better approach was to write a Python module. The module automatically fetches the various XML documents, parses them and caches the results, both to memory and to disk.<br />
<br />
I won't repeat the instructions for installing and using the module - that information is already given in an ipython notebook that you can view at <a href="https://github.com/stevemarple/python-aurorawatchuk/blob/master/aurorawatchuk/examples/aurorawatchuk_api.ipynb">https://github.com/stevemarple/python-aurorawatchuk/blob/master/aurorawatchuk/examples/aurorawatchuk_api.ipynb</a>. </div>
Steve Marplehttp://www.blogger.com/profile/15582257560382364498noreply@blogger.com0tag:blogger.com,1999:blog-252642453295590453.post-48264300519560266652016-12-28T18:14:00.000+00:002017-02-14T18:12:15.310+00:00AuroraWatch UK camera update #1<div dir="ltr" style="text-align: left;" trbidi="on">
<div dir="ltr">
<br />
<table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto; text-align: center;"><tbody>
<tr><td style="text-align: center;"><a href="https://astronomy-imaging-camera.com/wp-content/uploads/ASI174MC-MM_solar_moon_camera1.jpg" imageanchor="1" style="margin-left: auto; margin-right: auto;"><img border="0" height="500" src="https://astronomy-imaging-camera.com/wp-content/uploads/ASI174MC-MM_solar_moon_camera1.jpg" width="500" /></a></td></tr>
<tr><td class="tr-caption" style="text-align: center;">ZWO ASI 174MC colour USB 3.0 camera. Image from ZWO website.</td></tr>
</tbody></table>
Recently <a href="http://aurorawatch.lancs.ac.uk/" target="_blank">AuroraWatch UK</a> purchased a <a href="https://astronomy-imaging-camera.com/products/usb-3-0/asi174mc/" target="_blank">ZWO ASI174MC</a> colour USB 3.0 camera to (hopefully) record images of the aurora. The aim is to build an all-sky imager. To minimise the cost the camera will be connected to a <a href="https://www.raspberrypi.org/" target="_blank">Raspberry Pi</a> single-board computer and be located outside in a waterproof enclosure. This particular camera was chosen for several reasons:</div>
<ul style="text-align: left;">
<li>No mechanical shutter to wear out. </li>
<li>The large pixel size (5.86um) should give good sensitivity. </li>
<li>The sensor has an IR cut-off filter fitted (filters cannot be used with the fish-eye lens). </li>
<li>The manufacturer provides support to access the camera from Linux. </li>
</ul>
At present the Linux support from ZWO is limited to a software development kit (SDK) which provides a C language library and header file. This is sufficient for a Raspberry Pi to be able to interface with the camera. However my preferred approach is to write the image capture and control software in Python since that will simplify development and access to the <a href="http://aurorawatch.lancs.ac.uk/api-info/" target="_blank">AuroraWatch UK API</a>. I therefore decided to write a Python wrapper for the C library. This is not something I have done before and after a little experimentation with <a href="https://docs.python.org/2/library/ctypes.html" target="_blank">ctypes</a>, <a href="http://cython.org/" target="_blank">Cython</a> and <a href="http://cffi.readthedocs.io/en/latest/" target="_blank">CFFI</a> and I settled on using ctypes. You can find the source code for my Python wrapper on <a href="https://github.com/stevemarple/python-zwoasi" target="_blank">Github</a>. It's also available on <a href="https://pypi.python.org/pypi?name=zwoasi&version=0.0.5&:action=display" target="_blank">pypi</a> so you can install it simply by running "<span style="font-family: "courier new" , "courier" , monospace;">pip install zwoasi</span>".<br />
<br />
I've written a short demo program to illustrate how to access the camera from Python. The camera can operate in two modes. The first, snapshot mode, takes a single exposure. The second, video mode, continually takes exposures which can be downloaded from the camera as and when required. At first glance either mode would appear suitable for the purpose of taking a single exposure once a minute. My main concern is how to set the exposure correctly for an automated system which will run in a variety of different lighting levels (twilight, nighttime, and possibly even daytime). Fortunately this particularly camera provides an auto-exposure mode which can adjust the exposure time and gain settings according to the ambient light levels. It does this by analysing the image histogram after each exposure. The documentation doesn't explain the auto-exposure mode but I suspected that the camera would have to be operated in video mode for this to work. A quick test proved this to be correct. How well the auto-exposure algorithm works at night has yet to be discovered; since it is an astronomy camera I'm optimistic that it will be suitable and save me from having to implement an exposure control algorithm.<br />
<div class="separator" style="clear: both; text-align: center;">
</div>
<br />
<table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto; text-align: center;"><tbody>
<tr><td style="text-align: center;"><a data-flickr-embed="true" href="https://www.flickr.com/photos/stevemarple/31823392891/in/dateposted-ff/" style="margin-left: auto; margin-right: auto;" title="Sample ASI174MC image"><img alt="Sample ASI174MC image" height="500" src="https://c4.staticflickr.com/1/503/31823392891_09f98af549_z.jpg" width="500" /></a></td></tr>
<tr><td class="tr-caption" style="text-align: center;">Sample image from ASI174MC and Fukinon FE185C057HA-1 2/3" fisheye lens.</td></tr>
</tbody></table>
The complete control system is intended to timestamp and watermark images, change the recording cadence according to AuroraWatch UK status and solar elevation, record temperature and humidity, and control heating and fans.<br />
<br />
<br />
<br /></div>
Steve Marplehttp://www.blogger.com/profile/15582257560382364498noreply@blogger.com23tag:blogger.com,1999:blog-252642453295590453.post-22708156282450939232014-10-28T20:35:00.003+00:002014-11-29T22:52:08.654+00:00FTDI all-in-one serial converter, programmer, switch<div dir="ltr" style="text-align: left;" trbidi="on">
<br />
<table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto; text-align: center;"><tbody>
<tr><td style="text-align: center;"><a href="https://www.flickr.com/photos/stevemarple/15652130335" style="margin-left: auto; margin-right: auto;" title="FTDI all-in-one by Steve Marple, on Flickr"><img alt="FTDI all-in-one" height="409" src="https://farm6.staticflickr.com/5604/15652130335_79e8ecb0ec.jpg" width="500" /></a></td></tr>
<tr><td class="tr-caption" style="text-align: center;">FTDI all-in-one PCB layout.</td></tr>
</tbody></table>
<br />
<div>
There are a multitude of <a href="http://www.ftdichip.com/" target="_blank">FTDI</a> converters and breakout boards available but none of them exactly fitted my needs so this is my version tuned to suit my requirements. I'm publishing the design files in the hope they are useful for others. Like the <a href="https://www.sparkfun.com/" target="_blank">Sparkfun</a> <a href="https://www.sparkfun.com/products/9873" target="_blank">FTDI Basic Breakout</a> and the <a href="https://www.adafruit.com/" target="_blank">Adafruit</a> <a href="https://www.adafruit.com/products/284" target="_blank">FTDI Friend</a> it uses the <a href="http://www.ftdichip.com/" target="_blank">FTDI</a> <a href="http://www.ftdichip.com/Products/ICs/FT232R.htm" target="_blank">FT232R</a> but there are a number of differences, for instance it has a <a href="http://en.wikipedia.org/wiki/USB#Standard_connectors" target="_blank">USB type A</a> connector to plug directly into a USB host port.</div>
<div>
<br /></div>
<h3 style="text-align: left;">
Serial converter</h3>
<div>
The FTDI AIO provides a serial adaptor with the standard 6 pin FTDI pinout (TX, RX, CTS, DTR, VCC, and GND). A jumper block enables the VCC pin to be switched between +3.3V and +5V. A separate jumper enables the logic levels of the TX, RX, DTR and CTS signals to be set for +3.3V or +5V operation. The <a href="https://www.sparkfun.com/" target="_blank">Sparkfun</a> <a href="https://www.sparkfun.com/products/9873" target="_blank">FTDI Basic Breakout</a> can also be switched between +3.3V and +5V operation but does not allow for independent selection of VCC and the logic levels. This means when the Sparkfun FTDI Basic is set for +3.3V operation it's not compatible with the standard FTDI cables, which always supply +5V to the VCC pin. Whether that's a good thing or a bad thing depends on your needs; the FTDI AIO gives you both options.</div>
<div>
<br /></div>
<h3 style="text-align: left;">
AVR ISP programmer</h3>
<div>
Programming AVR microcontrollers, even with just a small bootloader, can be painfully slow and normally I'd use the <a href="http://www.atmel.com/tools/avrdragon.aspx" target="_blank">AVR Dragon</a>. There are situations however where it would be useful to have a low-cost USB programmer. A bit-banging AVR programmer is easily made and <a href="https://www.adafruit.com/" target="_blank">Adafruit</a> has an <a href="https://learn.adafruit.com/ftdi-friend/programming-blank-avrs" target="_blank">excellent tutorial</a> which explains how to do this.The FTDI all-in-one makes this convenient by providing the standard AVR ISP header. The defacto AVR standard is that the VCC pin is set to the same voltage as the logic-level signals. Unfortunately this practice isn't followed by +3.3V Arduino boards which always set VCC to +5V. For the widest compatibility the AVR ISP VCC pin can be set to either +3.3V or +5V with the logic levels of the !RESET, MISO, MOSI and SCK signals independently configured to either +3.3V or +5V.</div>
<div>
<br /></div>
<h3 style="text-align: left;">
On-board +3.3V voltage regulator</h3>
<div>
One of the features of the FT232R is that it has a built-in +3.3V regulator. This was used by older Arduinos to provide a +3.3V supply. However the current capability is limited to just 50mA which in many cases is not sufficient to power a microcontroller and all of its peripherals. The FTDI all-in-one can include an on-board voltage regulator with 250mA current output which should be sufficient to power an Arduino and several shields.</div>
<div>
<br /></div>
<h3 style="text-align: left;">
Auxiliary switch and LED</h3>
<div>
One of the key features I wanted was the ability to add an auxiliary switch and LED to the Raspberry Pi. Although the Pi has GPIO the <a href="http://blog.stevemarple.co.uk/search/label/AuroraWatchNet" target="_blank">AuroraWatchNet</a> magnetometers normally have a <a href="http://shop.ciseco.co.uk/slice-of-radio-wireless-rf-transciever-for-the-raspberry-pi/" target="_blank">radio</a> fitted to this connector, which prevents it being easily used for other purposes. I worked around this by wiring a switch and LED to a Sparkfun FTDI Basic Breakout but I wanted a single PCB which plugged directly into one of the Pi's USB sockets.</div>
<div>
<br /></div>
<div>
Why would you want a switch? There are probably many uses. Mine is that I run one of the AuroraWatchNet magnetometers and I wanted a way to conveniently signal when I am going to cut the grass, or make some other magnetic disturbance in the garden.The data logging software, which runs on a Raspberry Pi, checks for the existence of a file which signals data quality might be bad. When the file is present data is logged to a separate set of files which aren't used by the <a href="http://aurorawatch.lancs.ac.uk/" target="_blank">AuroraWatch UK</a> alerts system. While it's no bother for me to create the file by logging into the Pi I needed something simpler for other members of the family, so I wanted to wire up a switch. A separate daemon monitors the serial port to detect if the switch is pressed, if so the LED is turned on and the semaphore file created. If the switch is pressed again the file is removed and the LED turned off. The LED actually tracks the state of the file being present so it is possible to override the monitor process by creating or removing the semaphore file by other means.</div>
<div>
<br /></div>
<h3 style="text-align: left;">
Breakout board</h3>
<div>
All of the GPIO pins from the FTDI FT232R are conveniently broken out to pads spaced at 0.1" pitch. Also included are pads for the filtered +5V supply obtained from USB and +3.3V from the on-board regulator (if fitted, otherwise from the FT232R's built-in regulator).</div>
<div>
<br /></div>
<h3 style="text-align: left;">
Why use the FT232R?</h3>
<div>
Given the recent controversy regarding FTDI and Windows drivers you might wonder why I am using the FT232R instead of some other device. I'd probably prefer to use the <a href="http://www.microchip.com/" target="_blank">Microchip</a> <a href="http://www.microchip.com/MCP2200" target="_blank">MCP2200</a> USB UART converter for this application as it works well with my <a href="http://blog.stevemarple.co.uk/2013/02/calunium-version-21.html" target="_blank">Calunium</a> Arduino clone and it is available in a SOIC package which is easier to work with at home. Unfortunately its not possible to read the CTS line with the USB CDC driver which rules it out.<br />
<br />
<h3 style="text-align: left;">
Open-source hardware</h3>
The Eagle PCB design files are available on <a href="https://github.com/stevemarple/FTDI_AIO" target="_blank">Github</a> and are licensed under the <a href="http://creativecommons.org/licenses/by-sa/3.0/" target="_blank">Creative Commons Attribution-ShareAlike 3.0</a> (CC BY-SA 3.0) license.<br />
<br />
The PCB layout is not yet tested, although I did breadboard and test all of the functionality. I'll be sending the design off for manufacture in the next few days, it's likely I'll have a few boards spare.<br />
<br />
<b>Update:</b> the first boards have been assembled and tested; all features work as planned. The bit-banging AVR ISP programmer is even slower than I feared, taking about 10 minutes to read the 4Kb EEPROM on the Atmel ATmega1284P on one of my <a href="http://blog.stevemarple.co.uk/search/label/Calunium" target="_blank">Calunium</a> boards.</div>
<div>
<br /></div>
</div>
Steve Marplehttp://www.blogger.com/profile/15582257560382364498noreply@blogger.com0tag:blogger.com,1999:blog-252642453295590453.post-45816934595911547992014-09-12T21:36:00.000+01:002015-09-01T09:11:56.736+01:00Why Kp is a poor indicator for auroral alerts<div dir="ltr" style="text-align: left;" trbidi="on">
<h3>
What is Kp?</h3>
<div>
Kp is a an index to represent the planetary geomagnetic activity. The name originates from "planetarische Kennziffer" (German for planetary index). It has 10 main levels, numbered from 0 (no geomagnetic activity) to 9 (extreme geomagnetic activity). The intermediate ranges are subdivided with a + or - suffix, making 28 levels in total (0, 0+, 1-, 1, 1+, 2-, 2, 2+, ... 8, 8+, 9-, 9).<br />
<br />
The Kp index is calculated for three-hourly intervals, beginning at UT midnight. To calculate Kp the daily "solar-quiet" (Sq) variation is removed from the measurements of magnetic field strength. Then the difference between the largest and smallest values is computed. By looking up the difference in a conversion table the local K index can be found. Stations at different latitudes (or more properly magnetic latitudes) have different conversion tables.<br />
<br />
There is a pronounced daily variation in the K index at a single station, with intervals close to local midnight being substantially more disturbed in comparison with those centered on local noon; see <a href="http://www-app3.gfz-potsdam.de/kp_index/conv_tab.html">http://www-app3.gfz-potsdam.de/kp_index/conv_tab.html</a>. For this reason the planetary Kp index is computed from magnetic variations recorded by 13 magnetometers located around the world.<br />
<br /></div>
<h3>
Why Kp is a poor indicator for auroral alerts</h3>
<div>
The Kp index only indicates geomagnetic activity within a 3 hour interval, which is too long to be useful for auroral alerts. Geomagnetic conditions can have recovered to a calm state long before the next Kp index is computed. This is particularly true for aurora caused by a substorm, as the <i>entire</i> substorm cycle (including the growth, expansion and recovery phases) is typically 2 to 4 hours duration; see <a href="http://www.ann-geophys.net/29/2031/2011/angeo-29-2031-2011.html">http://www.ann-geophys.net/29/2031/2011/angeo-29-2031-2011.html</a>.</div>
<div>
<br /></div>
<div>
It must also be noted that the official Kp values are not available in real-time. Not only must the data from the 13 stations be collected the quiet-day curves for each of the magnetometers must be calculated for that calendar month, a process which cannot begin until the month ends. Therefore any references to Kp for the current month will be to unofficial estimates. The most reliable estimate of Kp is probably obtained from NOAA, which is derived from a worldwide distribution of magnetometers, see <a href="http://www.swpc.noaa.gov/products/planetary-k-index">http://www.swpc.noaa.gov/products/planetary-k-index</a>.<br />
<br />
<i>In summary: Kp is not available in real-time. The 3 hour interval means current auroral activity may be much lower than an estimated Kp value suggests.</i><br />
<br /></div>
<h3>
Estimated Kp can differ significantly from the local K index</h3>
As noted above, the pronounced daily variation of the K index means that there can be considerable difference between the average planetary geomagnetic activity (Kp) and that observed locally (K). The plot below shows the difference between the local K index measured by the <a href="http://www.bgs.ac.uk/" target="_blank">British Geological Survey</a> magnetometer at <a href="http://www.geomag.bgs.ac.uk/operations/eskdale.html" target="_blank">Eskdalemuir</a> and <a href="http://www.gfz-potsdam.de/forschung/ueberblick/departments/department-2/erdmagnetfeld/services/kp-index/download/" target="_blank">Kp</a>.<br />
<br />
<table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto; text-align: center;"><tbody>
<tr><td style="text-align: center;"><a href="http://www.flickr.com/photos/stevemarple/13311960204/" style="margin-left: auto; margin-right: auto;" title="Difference between Eskdalemuir local K index and Kp by Steve Marple, on Flickr"><img alt="Difference between Eskdalemuir local K index and Kp" src="http://farm8.staticflickr.com/7274/13311960204_1a1385af42.jpg" height="375" width="500" /></a></td></tr>
<tr><td class="tr-caption" style="text-align: center;">Difference between K index measured at Eskdalemuir and Kp.</td></tr>
</tbody></table>
<br />
The plot shows that in general there is good agreement but at certain times the K index can be higher or lower than Kp.<br />
<br />
<i>In summary: global measurements of Kp are not always valid indications of regional geomagnetic activity.</i><br />
<br />
<h3 style="text-align: left;">
What measurements should be used?</h3>
<div>
<div style="text-align: left;">
One of the better measurements is using the K index as long as it is from a nearby magnetometer or one on a similar geomagnetic longitude. It is still limited by the 3 hour resolution of the K index so it will not always be accurate for real-time measurements.</div>
</div>
<br />
An alternative measurement is to consider the rate of change of the magnetic field, commonly referred to by its mathematical notation dB/dt ("D B by D T"). This measurement is of particular interest to operators of pipelines and long power distribution networks since it indicates the levels of geomagnetically-induced currents which might be expected. British Geological Survey publish real-time <a href="http://www.geomag.bgs.ac.uk/data_service/space_weather/geoelectric.html">dB/dt measurements</a>. These measurements require magnetically quiet sites as human interference is likely to cause sudden spikes with a high dB/dt value, limiting its usefulness to observatory measurements.<br />
<br />
<a href="http://aurorawatch.lancs.ac.uk/">AuroraWatch UK</a> publishes a real-time auroral activity measurement. It is the H component deviation, which is the difference between the current H component magnetic field strength and the expected value for the time of day taken from the "quiet-day curve". Unlike Kp this measurement is computed hourly and so can indicate a return to low activity values sooner than Kp. An accurately fitted quiet day curve is required to make the deviation measurement.<br />
<br />
<h3>
</h3>
<h3>
Data credits</h3>
<a href="http://www.geomag.bgs.ac.uk/operations/eskdale.html" target="_blank">Eskdalemuir</a> K indices were downloaded from the <a href="http://www.bgs.ac.uk/" target="_blank">British Geological Survey</a>, <a href="https://www.bgs.ac.uk/discoverymetadata/13480122.html">https://www.bgs.ac.uk/discoverymetadata/13480122.html</a>. Kp data was downloaded from <a href="http://www.gfz-potsdam.de/en/home/">GFZ-Potsdam</a>, <a href="ftp://ftp.gfz-potsdam.de/pub/home/obs/kp-ap/tab/">ftp://ftp.gfz-potsdam.de/pub/home/obs/kp-ap/tab/</a>.</div>
Steve Marplehttp://www.blogger.com/profile/15582257560382364498noreply@blogger.com1tag:blogger.com,1999:blog-252642453295590453.post-18180293444307133552014-05-17T19:55:00.001+01:002014-05-17T19:55:51.692+01:00Calunium v2.1a<div dir="ltr" style="text-align: left;" trbidi="on">
I've made a few minor improvements to the <a href="http://blog.stevemarple.co.uk/search/label/Calunium" target="_blank">Calunium</a> design, partly to satisfy compatibility between Calunium and <a href="http://arduino.cc/" target="_blank">Arduino</a> shields and partly to fix a few minor hardware errors.<br />
<br />
<h3 style="text-align: left;">
Disconnect VCC from the ISP header</h3>
<div>
The ISP header is a 6 pin male header which can be used to program the microcontroller, such as to upload the bootloader. On Calunium the VCC pin of this header is wired to the same supply voltage used by the microcontroller, ie +3.3V or +5V depending upon the operating volatge selected. This makes sense, especially when used when programmers such as Atmel's <a href="http://www.atmel.com/tools/avrdragon.aspx" target="_blank">AVR Dragon</a> which use the pin to set the HIGH voltage level on the SCK and MOSI control lines. Unfortunately it seems that on Arduino devices (such as the <a href="http://arduino.cc/en/Main/ArduinoEthernetShield" target="_blank">Ethernet shield</a>) this pin is always wired to +5V. This is true even for the latest hardware versions which should be aware of the operating voltage via the IOREF pin. To avoid the possibility of connecting the +3.3V and +5V supplies I added a jumper. For normal operation the shunt can be omitted. When using an ISP header programmer that needs to know the operating voltage a shunt can be added.</div>
<div>
<br /></div>
<h3 style="text-align: left;">
Pull-down resistor on D13 FET</h3>
Calunium uses a field-effect transistor (FET) to switch on the D13 LED. This is to avoid load on the D13 SCK line when it is used for SPI communications. I learnt this trick from <a href="http://www.freetronics.com/">Freetronics</a> who <a href="http://www.freetronics.com/collections/arduino/products/kitten">do something similar</a> with their range of clones. In normal use, when D13 is an output, everything works fine. However, when it is an input, such as during RESET state or bootloader programming, the FET gate can float to potential such that the LED is switched on. A 1 megaohm pull down resistor connected to D13 is sufficient to prevent this happening without loading the SCK line. It also helps prevent damage by <a href="http://en.wikipedia.org/wiki/Electrostatic_discharge" target="_blank">electrostatic discharge</a>. The latest Calunium design now has room to add this 1 megaohm resistor. On older boards an 0805 surface mount resistor can be fitted between the gate and source of the 2N7000 FET, or alternatively between the D13 and adjacent ground pin.<br />
<br />
<h3 style="text-align: left;">
Auto-reset behaviour corrected for MCP2200 USB interface</h3>
I recently added the option of fitting the <a href="http://www.microchip.com/wwwproducts/devices.aspx?dDocName=en546923" target="_blank">MCP2200</a> USB-serial interface. Despite suggestions to the contrary this useful USB device <a href="http://blog.stevemarple.co.uk/2013/01/uart-usb-converter-for-mixed-33v5v.html" target="_blank">can automatically reset the Arduino</a> when the device is opened. This auto-reset behaviour as it is known is used to activate the bootloader when uploading sketches. The intention was that auto-reset could be disabled by removing a shunt from the AUTO RESET jumper. Unfortunately an error in the schematic meant that auto-reset could only be disabled when using the FTDI interface, not the MCP2200 option. The latest version corrects this error.<br />
<br />
<h3 style="text-align: left;">
Silkscreen label corrected</h3>
On <a href="http://blog.stevemarple.co.uk/2013/02/calunium-version-21.html" target="_blank">Calunium v2.1</a> a jumper was added to select FTDI or USB power. Due to a rotation of the component the correct position for the shunt is opposite to that indicated by the silkscreen label. The latest version corrects the text.<br />
<br />
<h3>
Open source</h3>
<div>
The Eagle PCB design files for Calunium are available on <span style="color: red;"><a href="https://github.com/stevemarple/Calunium" target="_blank">Github</a></span> and are licensed under the <a href="http://creativecommons.org/licenses/by-sa/3.0/" target="_blank">Creative Commons Attribution-ShareAlike 3.0 Unported License</a>. Files to allow Calunium to be used with the Arduino 1.0 IDE are available on <span style="color: red;"><a href="https://github.com/stevemarple/Calunium" target="_blank">Github</a></span> and are licensed under the <a href="http://www.gnu.org/licenses/gpl-2.0.html" target="_blank">Gnu General Public License v2</a>.</div>
<br /></div>
Steve Marplehttp://www.blogger.com/profile/15582257560382364498noreply@blogger.com0tag:blogger.com,1999:blog-252642453295590453.post-22996925685334553062014-03-24T23:04:00.000+00:002014-03-25T22:08:15.546+00:00Power over ethernet (PoE) magnetometer and cloud detector<div dir="ltr" style="text-align: left;" trbidi="on">
<table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto; text-align: center;"><tbody>
<tr><td style="text-align: center;"><a href="https://www.flickr.com/photos/stevemarple/13339933503/" style="margin-left: auto; margin-right: auto;" title="Combined magnetometer and cloud detector by Steve Marple, on Flickr"><img alt="Combined magnetometer and cloud detector" height="333" src="https://farm8.staticflickr.com/7036/13339933503_fa5ac7e6e9.jpg" width="500" /></a></td></tr>
<tr><td class="tr-caption" style="text-align: center;">Combined magnetometer and cloud detector.</td></tr>
</tbody></table>
<br />
<br />
<div dir="ltr" style="text-align: left;" trbidi="on">
To improve the performance and stability of the AuroraWatchNet magnetometers I recently began experimenting with a power over ethernet (PoE) version. With restrictions on power consumption lifted <a href="http://blog.stevemarple.co.uk/2014/02/performance-comparison-of-power-over.html" target="_blank">considerable performance improvements are possible</a>. As a result I have developed the magnetometer hardware specifically to support a power over ethernet version. Another instrument I've been developing is a <a href="http://blog.stevemarple.co.uk/search/label/cloud%20detector" target="_blank">cloud detector</a>. This too should benefit from a power over ethernet version. One problem I encountered was with dew settling on the sensor but fitting a heater is incompatible with battery-powered operation. Since both the magnetometer and cloud detector use almost the same hardware I decided to design an Arduino-compatible 'shield' that could be used to support both systems.<br />
<br />
<h3 style="text-align: left;">
Combined magnetometer and cloud detector hardware</h3>
<table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto; text-align: center;"><tbody>
<tr><td style="text-align: center;"><a href="https://www.flickr.com/photos/stevemarple/13326778455/" style="margin-left: auto; margin-right: auto;" title="Combined magnetometer and cloud detector PCBs by Steve Marple, on Flickr"><img alt="Combined magnetometer and cloud detector PCBs" height="333" src="https://farm8.staticflickr.com/7234/13326778455_d5b600b86b.jpg" width="500" /></a></td></tr>
<tr><td class="tr-caption" style="text-align: center;">Printed circuit boards for the sensor shield (left), the IR or humidity sensor (top right)<br />
and fluxgate magnetometer (bottom right). Click for annotated version.</td></tr>
</tbody></table>
The complete system requires six circuit boards (five for the wireless version). The first is the microcontroller board, I use my <a href="http://blog.stevemarple.co.uk/search/label/Calunium">Calunium</a> Arduino clone. I hope that in future it will be possible to use an off-the-shelf Arduino Mega2560 instead but the current firmware relies on Calunium's real-time clock to generate the hardware interrupts which control the sampling interval. There is also a sensor shield, the Arduino Ethernet shield (omitted on the battery-powered wireless version), and one board for each of the sensors (fluxgate magnetometer, IR temperature and humidity).<br />
<br />
The sensor shield is based on the <a href="https://github.com/stevemarple/AuroraWatchNet/tree/master/hardware/FLC100_shield/shield_v1.0" target="_blank">existing design</a> and retains the option of battery-powered operation with radio communication. The magnetometer sensor and analogue-to-digital converter must be powered at 5V, which requires a level-shifting circuit when the microcontroller is powered from 3.3V, which is the case when operating from batteries. For power over ethernet use the microcontroller must also operate at 5V for compatibility with the <a href="http://arduino.cc/en/Main/ArduinoEthernetShield" target="_blank">Arduino Ethernet shield</a> so the level shifting is not required. It is kept however as it provides buffering between the two circuit boards; 1.5m is a considerable distance for an <a href="http://en.wikipedia.org/wiki/I%C2%B2C" target="_blank">I2C bus</a>. The cloud detector uses a non-contact infra-red temperature sensor operating at 3.3V so a level-shifting circuit is required for PoE operation where the microcontroller is connected to 5V. (I've ignored the fact that a 5V version of the sensor exists since it isn't readily available in the UK). The sensor shield allows a humidity sensor to be connected so that estimates of the clear sky and cloudy sky temperatures can be made. As before, an on-board LM61 temperature sensor monitors the system temperature. The new sensor shield also adds a header to fit an <a href="http://www.embeddedadventures.com/as3935_lightning_sensor_module_mod-1016.html" target="_blank">Embedded Adventures lightning sensor module</a>. I don't have one of these at the moment so I can't be certain it will work and there is no software support for it in the existing firmware. Fitting it at the same time as the cloud detector sensors will require <a href="http://proto-pic.co.uk/break-away-headers-long/">long break-away headers</a> to be soldered to the bottom of the lightning sensor module.<br />
<br />
<table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto; text-align: center;"><tbody>
<tr><td style="text-align: center;"><a href="https://www.flickr.com/photos/stevemarple/13390786613/" style="margin-left: auto; margin-right: auto;" title="Fluxgate magnetometer sensor PCB by Steve Marple, on Flickr"><img alt="Fluxgate magnetometer sensor PCB" height="320" src="https://farm3.staticflickr.com/2805/13390786613_5ddc2c5775_n.jpg" width="320" /></a></td></tr>
<tr><td class="tr-caption" style="text-align: center;">Fluxgate magnetometer sensor mounted on its sensor PCB.<br />
Click for annotated version.</td></tr>
</tbody></table>
<br />
The fluxgate sensor is fitted on <a href="https://github.com/stevemarple/AuroraWatchNet/tree/master/hardware/FLC100_shield/remote_v2">its own PCB</a> which contains the analogue to digital converter. For PoE operation a linear voltage regulator is used to convert from 9V to the 5V supply it requires. For battery operation a MAX619 DC-DC charge pump boosts the battery voltage to 5V. Almost all of the temperature variation can be removed by placing the PCB at the bottom of a 1m length of soil pipe. The pipe is buried to a depth of 0.85m with its axis vertical. In the magnetometer-only system the microcontroller, sensor shield and ethernet shield (or batteries for the wireless version) are fitted onto a wooden frame to hold them into the top part of the tube. Positioning the rest of the system away from the fluxgate sensor helps to avoid unwanted effects from any ferro-magnetic components (such as the batteries), it also aids access and enables wireless data transmission.<br />
<br />
<table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto; text-align: center;"><tbody>
<tr><td style="text-align: center;"><a href="https://www.flickr.com/photos/stevemarple/13391063743/" style="margin-left: auto; margin-right: auto;" title="Humidity sensor PCB by Steve Marple, on Flickr"><img alt="Humidity sensor PCB" height="320" src="https://farm8.staticflickr.com/7455/13391063743_2169e7ac75_n.jpg" width="320" /></a></td></tr>
<tr><td class="tr-caption" style="text-align: center;">Honeywell HIH6131 humidity sensor mounte on its sensor PCB.<br />
The same PCB design is also used for the MLX90614 IR temperature sensor.<br />
Click for annotated version.</td></tr>
</tbody></table>
<br />
The enclosure design used for the magnetometer isn't suitable for the cloud detector so the prototype cloud detector used a <a href="http://uk.farnell.com/multicomp/g3113/box-abs-ip65-200x120x90mm/dp/1526655?Ntt=1526655">standard IP65 rated box</a>, with the IR temperature sensor pointing upwards to view the sky. The humidity sensor was fitted above a breather hole in the bottom of the box. This concept will continue to be used for the cloud detector; the IR temperature sensor and humidity sensor are fitted to separate PCBs in the top and bottom of the box. To minimise costs the IR temperature and humidity sensors use the same PCB design. The mechanical design of the cloud detector part is something I'd like to improve upon, particularly to reduce the number of PCBs used. However the differing sensor requirements may prevent this.<br />
<div>
<br /></div>
For a combined system I plan to use the soil pipe to house the fluxgate sensor but locate the rest of the electronics in a separate box following the design of the prototype cloud detector.<br />
<br />
<h3 style="text-align: left;">
Does it work?</h3>
<div>
I've not yet deployed a system outside but testing indicates the new printed circuit boards work as intended when used in power over ethernet mode. Battery-powered operation on this new version has not yet been tested.</div>
<div>
<br /></div>
<h3 style="text-align: left;">
Design files</h3>
<div>
The design files (hardware, firmware and software) are open source and can all be downloaded from the <a href="https://github.com/stevemarple/AuroraWatchNet">Github repository</a>. A PDF version of the user manual describing how to construct and operate the magnetometer can be downloaded from <a href="http://aurorawatch.lancs.ac.uk/manual.pdf">http://aurorawatch.lancs.ac.uk/manual.pdf</a>. At the time of writing only instructions to build the original FLC100 shield are included. Instructions to build the combined sensor shield described above will be added in due course.</div>
<br />
<br />
<h4 style="text-align: left;">
</h4>
</div>
</div>
Steve Marplehttp://www.blogger.com/profile/15582257560382364498noreply@blogger.com0tag:blogger.com,1999:blog-252642453295590453.post-33906246070605902152014-02-12T23:35:00.000+00:002014-02-12T23:35:04.088+00:00Performance comparison of power over ethernet (PoE) and battery magnetometers<div dir="ltr" style="text-align: left;" trbidi="on">
<h3 style="text-align: left;">
Introduction</h3>
I've been working to improve the performance and stability of the AuroraWatchNet magnetometers. It's apparent that both measurement noise and stability are considerably improved when the sensor is powered continually. Unfortunately with the sensor powered continually the batteries last for only a few weeks. I have therefore been testing a power over ethernet (PoE) version of the magnetometer. The hardware is essentially the same but the addition of the <a href="http://arduino.cc/en/Main/ArduinoEthernetShield" target="_blank">Arduino Ethernet shield</a> requires the microcontroller to operate from 5V. Operating voltage is easily changed on my <a href="http://blog.stevemarple.co.uk/search/label/Calunium" target="_blank">Calunium</a> microcontroller board. With the power restrictions lifted the sampling interval can also be reduced. The test system has been operating reliably for almost two months, sampling every 5 seconds. With some minor configuration changes sampling every second is possible although I am not convinced the trade-off in measurement noise is worthwhile.<br />
<br />
<h3>
Performance comparison</h3>
<div>
Below is a plot comparing one hour of data from the new power over ethernet system with two of the existing battery-powered wireless models already in operation. The PoE system is on the same site as LAN1 but located nearer parked cars. I chose this period because it was free of man-made disturbances. I adjusted the baselines so that the plots overlap.Only the H component of the magnetic field is shown.<br />
<br /></div>
<table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto; text-align: center;"><tbody>
<tr><td style="text-align: center;"><a href="http://www.flickr.com/photos/stevemarple/12486994313/" style="margin-left: auto; margin-right: auto;" title="Power over ethernet compared with existing AWN magnetometers by Steve Marple, on Flickr"><img alt="Power over ethernet compared with existing AWN magnetometers" src="http://farm4.staticflickr.com/3687/12486994313_6d05555212.jpg" /></a></td></tr>
<tr><td class="tr-caption" style="text-align: center;">Click for larger version.</td></tr>
</tbody></table>
<br />
The graph shows that the power over ethernet version has <i>much</i> smaller measurement errors; it can probably operate with ~0.1nT accuracy compared to ~10nT for the battery-powered version. It's such an improvement that I found the measurement accuracy was being limited by the available resolution of the analogue-to-digital converter. The battery-powered versions derive the sample value from the median of 15 samples (taken as a burst over 4 seconds) to reduce noise. To improve the resolution for the PoE version it operates by taking the mean of 15 samples. Further improvement to the resolution may be possible by taking advantage of the programmable gain amplifier that is built into the analogue-to-digital converter.<br />
<br />
Let's see how it compares against some observatory-grade measurements. The plot below shows the same interval but this time uses data from the <a href="http://geomag.bgs.ac.uk/operations/observatories.html" target="_blank">British Geological Survey</a> magnetometers at <a href="http://geomag.bgs.ac.uk/operations/eskdale.html" target="_blank">Eskdalemuir</a> and <a href="http://geomag.bgs.ac.uk/operations/hartland.html" target="_blank">Hartland</a>. I obtained the data from the <a href="http://spears.lancs.ac.uk/samnet/" target="_blank">SAMNET</a> data archive at Lancaster University, where the data has already been converted into HEZ magnetic coordinates. AuroraWatchNet also operates with HEZ magnetic coordinates, although usually the E and Z sensors are not present. As before, the baselines have been adjusted so that the plots overlap.<br />
<br />
<br />
<table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto; text-align: center;"><tbody>
<tr><td style="text-align: center;"><a href="http://www.flickr.com/photos/stevemarple/12486875635/" style="margin-left: auto; margin-right: auto;" title="PoE magnetometer compared with BGS magnetometers by Steve Marple, on Flickr"><img alt="PoE magnetometer compared with BGS magnetometers" src="http://farm8.staticflickr.com/7451/12486875635_ffe3e08274.jpg" height="375" width="500" /></a></td></tr>
<tr><td class="tr-caption" style="text-align: center;">Click for larger version.</td></tr>
</tbody></table>
<br />
<br />
The similarity between the different traces is striking. Some differences are to be expected since Eskdalemuir is approximately 140km north of Lancaster and Hartland is 350km SSW of Lancaster. This interval is interesting because it shows <a href="https://wiki.oulu.fi/display/SpaceWiki/Pi+2+pulsations" target="_blank">Pi2 pulsations</a>, starting at about 01:15 and ending around 01:30. The period of the pulsations are about 90 to 120 seconds.<br />
<br />
<h3 style="text-align: left;">
Conclusions</h3>
<div>
For a home-built citizen-science magnetometer which probably costs 25 times less than its observatory grade cousin I'm very happy with its performance. The detection of Pi2 pulsations means a low-cost magnetometer can now notify of <a href="http://en.wikipedia.org/wiki/Substorm" target="_blank">substorm</a> onset, not just the arrival of geomagnetic storms. A network of such devices has interesting possibilities for the study of magnetic field line resonances.</div>
<br />
<div>
You might wonder why anyone would want to buy an observatory grade instrument, there are good reasons. At present I am relying on the sensor manufacturer's calibration, whilst an observatory grade instrument would be supplied with an official calibration certificate. The observatory instrument would also have better long-term baseline stability, lower temperature variation and higher cadence. Calibration is an issue I hope to tackle at a later date. For space-weather monitoring only short-term variations are of interest and<br />
the better performance provided by an observatory system may not be needed.<br />
<br /></div>
<h3 style="text-align: left;">
Data credits</h3>
<span style="font-size: small;"><span style="font-weight: normal;">The Sub-Auroral Magnetometer Network data (<a href="http://spears.lancs.ac.uk/samnet/" target="_blank">SAMNET</a>) is operated by the Space Plasma Environment and Radio Science (<a href="http://spears.lancs.ac.uk/" target="_blank">SPEARS</a>) group, Department of Physics, <a href="http://www.lancaster.ac.uk/" target="_blank">Lancaster University</a>. </span></span>Hartland and Eskdalemuir data is provided courtesy of the <a href="http://www.geomag.bgs.ac.uk/" target="_blank">British Geological Survey</a>.<br />
<div>
<br /></div>
<div>
<span property="dct:title" xmlns:dct="http://purl.org/dc/terms/">AuroraWatchNet LAN1 magnetometer data</span> by <a href="http://blog.stevemarple.co.uk/" property="cc:attributionName" rel="cc:attributionURL" xmlns:cc="http://creativecommons.org/ns#">Steve Marple</a> is licensed under a <a href="http://creativecommons.org/licenses/by-nc-sa/4.0/" rel="license">Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International License</a>.<br />
Based on a work at <a href="http://aurorawatch.lancs.ac.uk/data/aurorawatchnet/lan1/" rel="dct:source" xmlns:dct="http://purl.org/dc/terms/">http://aurorawatch.lancs.ac.uk/data/aurorawatchnet/lan1/</a>.</div>
<div>
<br /></div>
<div>
<span property="dct:title" xmlns:dct="http://purl.org/dc/terms/">AuroraWatchNet LAN3 magnetometer data</span> by <a href="http://www.physics.lancs.ac.uk/" property="cc:attributionName" rel="cc:attributionURL" xmlns:cc="http://creativecommons.org/ns#">Department of Physics, Lancaster University, UK</a> is licensed under a <a href="http://creativecommons.org/licenses/by-nc-sa/4.0/" rel="license">Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International License</a>.<br />
Based on a work at <a href="http://aurorawatch.lancs.ac.uk/data/aurorawatchnet/lan3/" rel="dct:source" xmlns:dct="http://purl.org/dc/terms/">http://aurorawatch.lancs.ac.uk/data/aurorawatchnet/lan3/</a>.</div>
<div>
<br /></div>
</div>
Steve Marplehttp://www.blogger.com/profile/15582257560382364498noreply@blogger.com0tag:blogger.com,1999:blog-252642453295590453.post-26019376338996159292013-10-27T16:40:00.001+00:002013-10-27T16:40:49.045+00:00Magnetometer progress report: link indicator LEDI recently added a new feature to the magnetometer remote sensor unit firmware. To help indicate when it is within radio communication range of the base unit an LED is turned on at the start of a message communication. When the sensor unit receives the acknowledgement from the base unit of successful message receipt the LED is switched off. In normal operation the LED should blink briefly every 30 seconds, after each sampling interval. If the LED remains on it indicates a problem with the radio link. For the initial installation the sampling time can be reduced to 5 seconds for to obtain faster feedback of whether communication is successful or not.<br />
<div>
<br /></div>
<h3>
Improvements for battery-powered operation</h3>
<div>
Indicating link errors by using an LED is convenient during installation but the power wasted after installation is not compatible with battery-powered operation. To save power the LED is used only the first 15 minutes of operation. Only user-initiated reset actions (power on or reset switch pressed) cause the LED to be used, resets from by the watchdog timer or brown-out detector are ignored. The cause of the reset is detected by examining the microcontroller's status register (MCUSR).</div>
<div>
<br /></div>
<div>
<br /></div>
<h3>
Communication timeout feature added</h3>
I've also added a timeout which detects when communication has been lost for an extended period. The microcontroller system is rebooted in the hope that the error is recoverable. Loss of communication, along with low battery status, can also be sensed and reported by running the latest version of the data plotting software on the Raspberry Pi. If either error is detected a message can be sent via email, Twitter or Facebook.<br />
<br />
<br />Steve Marplehttp://www.blogger.com/profile/15582257560382364498noreply@blogger.com0tag:blogger.com,1999:blog-252642453295590453.post-80034982494258845742013-10-19T00:17:00.000+01:002013-10-19T00:17:09.056+01:00Cloud detector: a review of progress so far<h3>
The hardware</h3>
My cloud detector has been running outside for over 5 months now. Overall I'm very pleased with how it works. The battery-powered hardware is based on the <a href="http://blog.stevemarple.co.uk/search/label/AuroraWatchNet" target="_blank">AuroraWatchNet magnetometer</a> design, which uses my own <a href="http://blog.stevemarple.co.uk/search/label/Calunium" target="_blank">Calunium</a> microcontroller development board. The remote sensor board and fluxgate sensor are omitted. I've added a Melexis MLX90614 non-contact infra-red thermometer to measure the sky temperature. Clear skies should give low temperatures whilst clouds are expected to have warmer temperatures, although still colder than the ambient temperature. The MLX90614 also outputs the sensor temperature, which should be close to the ambient temperature. I also added a Honeywell HIH-6131-021 humidity sensor which has an I2C interface. Do not confuse with the similar sounding version which has an SPI interface! The HIH-6130 also provides and ambient temperature measurement. To avoid direct contact with water the humidity sensor is placed at the bottom of the enclosure with a hole underneath to expose it to air. This hole also functions as a breather hole and ensures that the internal pressure matches atmospheric pressure. Since this hole was drilled there have not been any more incidents of water ingress.<br />
<br />
The Calunium board is running a firmware is a modified version of the magnetometer firmware. It communicates with the Raspberry Pi data logger using an 868MHz radio link, with the same signed communication protocol used by AuroraWatchNet. This means I can use the data recording software from AuroraWatchNet. It also inherits the signed over-the-air firmware update capability. In principle it should be possible to combine both magnetometer and cloud detector functions in one unit.<br />
<br />
I started off using the 90° field-of-view version of the non-contact IR thermometer but later switched to theMLX90614ESF-DCH-000-TU-ND variant which features a 12° field of view. Its greater height meant it was easier to fit into the cable grommet housing. However I think the FOV is too narrow and I plan to switch back to the original sensor when I have time.<br />
<br />
<h3>
Data processing</h3>
The data processing and plotting now uses the <a href="http://blog.stevemarple.co.uk/search/label/auroraplot" target="_blank">auroraplot</a> library for <a href="http://www.python.org/" target="_blank">Python</a>, which I developed to process the AuroraWatchNet magnetometer data. On the graphs I plot the sky temperature measured by the non-contact IR thermometer and the ambient temperature measured by the humidity sensor. For monitoring purposes I also plot the sensor temperature of the IR thermometer (marked as "detector temperature"); this should be similar to the ambient temperature but its exposed position makes it more likely to undergo solar heating and radiative cooling. For good measure I also plot the system temperature, which is measured by an LM61 temperature sensor connected to the ATmega1284P's analogue to digital converter. This measurement is noisy but useful to check the system doesn't overheat on sunny summer days.<br />
<br />
The plan is to estimate the upper and lower bounds that I would expect for the sky temperature, and from that derive an estimate of the cloud cover. I initially expected that with complete cloud cover the sky temperature would match that the <a href="http://en.wikipedia.org/wiki/Lifted_condensation_level" target="_blank">lifted condensation level</a>, which I estimate using the ambient temperature and relative humidity. I soon saw temperatures higher than the LCL temperature. Research literature indicates that the clouds can act as a mirror at long IR wavelengths and thus the expected temperature should include an effect of ground temperature too. On the plots this is shown as the effective cloud temperature. The clear sky temperature is derived from results found in research literature but none of the equations tried so far have been a great match. Other researchers have fitted parameters for their specific location (including altitude) by comparison with visual measurements. I have yet to try this but daily measurements are made at the nearby <a href="http://www.lancaster.ac.uk/lec/about-us/facilities/hazelrigg-weather-station/" target="_blank">Hazelrigg Weather Station</a>. The graphs are too cluttered for production use but help me to understand what is happening.<br />
<br />
<h3>
Example plots</h3>
<div>
Below are a selection of plots. You can see the entire archive (from 2013-07-14) at <a href="http://aurorawatch.lancs.ac.uk/testing/cloudwatch/test2/">http://aurorawatch.lancs.ac.uk/testing/cloudwatch/test2/</a>.</div>
<div>
<br /></div>
<table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto; text-align: center;"><tbody>
<tr><td style="text-align: center;"><a href="http://www.flickr.com/photos/stevemarple/10350907535/" style="margin-left: auto; margin-right: auto;" title="cloud_detector_20130903 by Steve Marple, on Flickr"><img alt="cloud_detector_20130903" height="375" src="http://farm6.staticflickr.com/5546/10350907535_6d4ee7c0f5.jpg" width="500" /></a></td></tr>
<tr><td class="tr-caption" style="text-align: center;">Cloud detector data for 2013-09-03. The asterisk (*) indicates derived parameters (ie not directly measured).</td></tr>
</tbody></table>
The figure above include most of the effects that can be identified. After midnight there is thick cloud, which later clears for short periods. At around 0300 UT I think it must have rained, the sky temperature is almost identical to the detector temperature and the variability is much reduced. Whilst the sensor is wet no sensible conclusions about cloud cover can be drawn. At about 0700 UT the sensor clears of water and the sky temperature falls indicating clearer skies. The rest of the day is dry, with heavy cloud cover until 1400 UT. At around 1700 UT the skies are completely clear. The clear sky estimate is too low for this time. At 2000 UT the clouds return, with some clear patches.<br />
<div>
<br />
<br />
<table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto; text-align: center;"><tbody>
<tr><td style="text-align: center;"><a href="http://www.flickr.com/photos/stevemarple/10351982655/" style="margin-left: auto; margin-right: auto;" title="Cloud detector, 2013-09-28 by Steve Marple, on Flickr"><img alt="Cloud detector, 2013-09-28" height="375" src="http://farm8.staticflickr.com/7314/10351982655_5a38967c22.jpg" width="500" /></a></td></tr>
<tr><td class="tr-caption" style="text-align: center;">Cloud detector data for 2013-09-28 showing the effect of clear skies.</td></tr>
</tbody></table>
The figure above shows an almost completely cloud-free day. The higher variability in the sky temperature measurements between 1200-1600 UT corresponds to similar effects in the humidity data (shown below); both are direct measurements made by different sensors.<br />
<br />
<table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto; text-align: center;"><tbody>
<tr><td style="text-align: center;"><a href="http://www.flickr.com/photos/stevemarple/10351982034/" style="margin-left: auto; margin-right: auto;" title="Relative humidity, 2013-09-28 by Steve Marple, on Flickr"><img alt="Relative humidity, 2013-09-28" height="234" src="http://farm3.staticflickr.com/2828/10351982034_f893d2d1fb.jpg" width="500" /></a></td></tr>
<tr><td class="tr-caption" style="text-align: center;">Humidity data for 2013-09-28. Notice how the higher variability occurs at the same times as the sky temperature measurements.</td></tr>
</tbody></table>
<br />
<h3>
Dew</h3>
<div>
During summer operation dew has not been a problem but now that the nights are colder I have noticed effects which I believe are due to the formation of dew on the sensor. In the first plot below the sky temperature apparently rises from around 2230 UT. The rise is smooth and during this period the ambient temperature is <i>falling</i>, which overnight is often a sign of clear skies<i>.</i> After midnight (second plot) the sky temperature increases slightly before falling sharply around 0630 UT. Sunrise on this day was 0622 UT.</div>
<div>
<br /></div>
<table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto; text-align: center;"><tbody>
<tr><td style="text-align: center;"><a href="http://www.flickr.com/photos/stevemarple/10352287124/" style="margin-left: auto; margin-right: auto;" title="Cloud detector, 2013-10-05 by Steve Marple, on Flickr"><img alt="Cloud detector, 2013-10-05" height="240" src="http://farm4.staticflickr.com/3812/10352287124_c815dc7fd9_n.jpg" width="320" /></a><a href="http://www.flickr.com/photos/stevemarple/10352290206/" title="Cloud detector, 2013-10-06 by Steve Marple, on Flickr"><img alt="Cloud detector, 2013-10-06" height="240" src="http://farm8.staticflickr.com/7414/10352290206_17316fc419_n.jpg" width="320" /></a></td></tr>
<tr><td class="tr-caption" style="text-align: center;">This pair of plots is believed to show dew formation on the sensor.</td></tr>
</tbody></table>
<h3>
<br /></h3>
<h3>
Conclusions</h3>
</div>
<div>
I know from the informal comparisons with visual cloud cover that I regularly make the cloud detector does generally work very well. It does not function during wet periods. As anticipated dew is becoming a problem during the colder nights and a dew heater will be required for reliable winter operation. Future development will be to add a heater, which will need a wired connection to the cloud detector. Once a wired connection is made the radio link appears superfluous so I plan to investigate the options for power-over-ethernet. The Arduino ethernet shield is one possibility, although its compatibility with 3.3V operation has not been established.</div>
Steve Marplehttp://www.blogger.com/profile/15582257560382364498noreply@blogger.com2tag:blogger.com,1999:blog-252642453295590453.post-45099818358134592512013-09-26T21:59:00.000+01:002013-10-18T21:16:42.675+01:00Auroraplot: data processing software for AuroraWatchNetWith the dispatch of the AuroraWatch schools' magnetometers imminent I have implemented a <a href="http://www.python.org/" target="_blank">Python</a> toolkit to process the data. The <a href="http://www.numpy.org/" target="_blank">numpy</a> and <a href="http://matplotlib.org/" target="_blank">matplotlib</a> modules are used extensively. The toolkit provides an <a href="http://en.wikipedia.org/wiki/Api" target="_blank">API</a> to load data and perform various processing actions on it, such as plotting data. The concept is influenced by my previous <a href="http://www.research.lancs.ac.uk/portal/en/publications/a-multiinstrument-data-analysis-toolbox(a7403b2b-6aef-49c0-8ab4-75fa4a39b810).html" target="_blank">Multi-instrument Analysis toolbox for Matlab</a>. In addition to the loading and processing of magnetic field data <auroraplot also="" and="" approach="" as="" battery="" by="" code="" from="" house-keeping="" information="" loading="" magnetometers="" object-oriented="" of="" operating="" processing="" such="" supports="" temperatures.="" the="" used="" voltage="">auroraplot</auroraplot> allows other data types to be added later.<br />
<br />
<h3>
Loading data</h3>
Data can be loaded with arbitrary start and end times very simply:<br />
<code><br />
md = ap.load_data('AURORAWATCHNET', 'LAN1', 'MagData',<br />
np.datetime64('2013-09-20T12:00:00+0000'),<br />
np.datetime64('2013-09-21T12:00:00+0000'))<br />
</code><br />
<br />
In this case the selected portion of data crosses midnight and two data files must be loaded, concatenated and trimmed to get the desired time range. This is performed automatically by <code>auroraplot</code>, the user need not be concerned with the format of the files or where they are located. It is even possible for the files to be downloaded on-the-fly using FTP or HTTP transfer protocols.<br />
<br />
<code>load_data</code> returns an object (of type <code>MagData</code>) to the user containing the actual magnetic field data and various other metadata, such as a timestamp for each sample and the data units. Each object can store more multiple data <i>channels</i> but all data points must share the same timestamps, be of the same type and share the same units. Therefore it is not possible to store operating temperatures (units °C) in an object holding magnetic field strength (units tesla). The operating temperature data can be accessed as:<br />
<code><br />
td = ap.load_data('AURORAWATCHNET', 'LAN1', 'TemperatureData', <br />
np.datetime64('2013-09-20T12:00:00+0000'),<br />
np.datetime64('2013-09-21T12:00:00+0000'))<br />
</code><br />
<br />
Battery voltage (data type <code>VoltageData</code>) can be accessed in a similar same way.<br />
<br />
<h3>
Plotting data</h3>
High-level plot functions enable the data be be plotted very simply, for the magnetic field data loaded previously<br />
<code><br />
md.plot()<br />
</code><br />
<br />
will produce a matplotlib figure with a title and the axes labelled with the correct units. Temperature and voltage data are plotted in the same way.<br />
<br />
I have created some tools to make working with numpy's datetime64 and timedelta64 objects more convenient, including rounding functions (<code>round</code>, <code>ceil</code> and <code>floor</code>) which round to an interval. They are useful for finding the start of an hour, or the end of a day. I have also created <code>Locator</code> and <code>Formatter</code> classes to sensibly label time axes using <code>datetime64</code> times and <code>timedelta64</code> intervals. Tick marks are located on the nearest second, minute, hour, day, month or year boundary (or multiple thereof) depending on the time interval being displayed. Thanks to matplotlib's structure the labels are automatically regenerated with the most appropriate time units when a plot is zoomed.<br />
<br />
<h3>
Other operations</h3>
<h4>
Quiet-day curves</h4>
Other operations include the generation of <i>quiet-day curves</i>. These are the curves from which we measure geomagnetic activity and are of critical importance for AuroraWatch UK. There are are not flat but have a daily variation caused by the <a href="http://en.wikipedia.org/wiki/Equatorial_electrojet" target="_blank">equatorial electrojet</a>. The empirical algorithm selects the days (typically 5) with the least geomagnetic activity. A truncated Fourier series is used to guarantee that the quiet-day curves are cyclic, with the start and end points having the same magnitude and slope. This is essential otherwise our rolling plots would show up the discontinuities in the QDC at midnight, and would falsely cause step changes in the geomagnetic activity. An example QDC is shown below.<br />
<br />
<table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto; text-align: center;"><tbody>
<tr><td style="text-align: center;"><a href="http://www.flickr.com/photos/stevemarple/9953317183/" style="margin-left: auto; margin-right: auto;" title="qdc by Steve Marple, on Flickr"><img alt="qdc" height="375" src="http://farm8.staticflickr.com/7298/9953317183_812e21789b.jpg" width="500" /></a></td></tr>
<tr><td class="tr-caption" style="text-align: center;">Quiet-day curve for magnetometer at Lancaster ,UK. This is derived from recorded data<br />
and clearly shows the Sq current system caused by the equatorial electrojet.</td></tr>
</tbody></table>
<br />
From this we can see that even on a geomagnetically quiet day we would expect a 30nT variation in field strength seen by the magnetometer. The AuroraWatch threshold for minor geomagnetic activity is 50nT so this shows the importance of using a quiet-day curve instead of a flat line when calculating geomagnetic activity.<br />
<br />
<h4>
Stack plots</h4>
Stack plots (also called <i>magnetograms</i>) are a convenient representation for magnetic field data from a set of magnetometers separated in latitude.Data from the northernmost instruments is placed at the top and that from the southernmost at the bottom. An example stackplot is shown below:<br />
<br />
<table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto; text-align: center;"><tbody>
<tr><td style="text-align: center;"><a href="http://www.flickr.com/photos/stevemarple/9955300805/" style="margin-left: auto; margin-right: auto;" title="20130920 by Steve Marple, on Flickr"><img alt="20130920" height="375" src="http://farm8.staticflickr.com/7312/9955300805_d2f1d7757d.jpg" width="500" /></a></td></tr>
<tr><td class="tr-caption" style="text-align: center;">Stackplot showing data from two Lancaster stations and from Ormskirk.</td></tr>
</tbody></table>
The magnetometer at Ormskirk is operated by the Met Office as part of a test. The stackplots will be more interesting as the network grows.<br />
<br />
<h3>
Open source</h3>
<div>
The source code is available under a BSD-type license from <a href="https://github.com/stevemarple/auroraplot" target="_blank">Github</a>.You will need python, along with the numpy (version 1.7), matplotlib and scipy python modules.auroraplot has been tested under Debian Linux (64 bit version) and Raspbian on the Raspberry Pi.</div>
<br />
<div>
<br /></div>
Steve Marplehttp://www.blogger.com/profile/15582257560382364498noreply@blogger.com0tag:blogger.com,1999:blog-252642453295590453.post-88720254491138425682013-08-08T16:03:00.000+01:002013-08-16T21:51:19.789+01:00Securing your SSH server against brute force password attacksIf you run an SSH server open to the whole internet you'll soon notice many attempts to break into your system using random username and password combinations. This 'noise' in the log files is annoying and might mean you miss an important message. If you are unlucky or use a common username and password your system might be compromised by one of these brute force attacks.<br />
<br />
One method to drastically reduce the number of cracking attempts is to restrict access to the SSH server by IP address. This can done be done either by editing the SSH config file or by using configuring <span style="font-family: Courier New, Courier, monospace;">iptables</span>. However this method requires valid users to have static IP addresses, something which most home users don't have. In many cases public key authentication is a better choice.<br />
<br />
<h3>
Use public key authentication</h3>
A better method to stop attackers from using random username and passwords is to switch off password authentication entirely and require public key authentication instead. Attackers will not be able to try random username/password combinations and will not create 'noise' in the log files. Valid users will still be able to connect.<br />
<br />
To describe how this can be set up assume we wish to protect a computer with a static IP address called <span style="font-family: Courier New, Courier, monospace;">server. W</span>e wish to access it from another system called <span style="font-family: Courier New, Courier, monospace;">laptop</span>, (which may be using a dynamic IP address). The account on <span style="font-family: Courier New, Courier, monospace;">server</span> that we will access is <span style="font-family: Courier New, Courier, monospace;">dave</span>. We'll assume that the SSH configuration file is <span style="font-family: Courier New, Courier, monospace;">/etc/ssh/sshd_config</span>. Adjust these names to suit your setup.<br />
<br />
Firstly, on <span style="font-family: Courier New, Courier, monospace;">laptop</span> create a public/private key pair.<br />
<blockquote class="tr_bq">
<span style="font-family: Courier New, Courier, monospace;">ssh-keygen -t dsa</span></blockquote>
<br />
A passphrase is not required although it is a good idea. On <span style="font-family: Courier New, Courier, monospace;">server</span> make sure a <span style="font-family: Courier New, Courier, monospace;">~/.ssh</span> directory exists with the correct permissions:<br />
<blockquote class="tr_bq">
<span style="font-family: Courier New, Courier, monospace;">mkdir ~dave/.ssh<br />
chmod go= ~dave/.ssh</span></blockquote>
<br />
Using your favourite editor (emacs of course) paste the contents of the public key into the <span style="font-family: Courier New, Courier, monospace;">~dave/.ssh/authorized_keys</span> file on <span style="font-family: Courier New, Courier, monospace;">server</span>. <a href="http://www.chiark.greenend.org.uk/~sgtatham/putty/" rel="nofollow" target="_blank">Putty</a> users may find the instructions at <a href="http://www.howtoforge.com/ssh_key_based_logins_putty" rel="nofollow" target="_blank">http://www.howtoforge.com/ssh_key_based_logins_putty</a> helpful. Check that you can log into <span style="font-family: Courier New, Courier, monospace;">server</span> without requiring your password. If you are going to configure <span style="font-family: Courier New, Courier, monospace;">server</span> remotely ensure you can log in without needing your password, otherwise you will lock yourself out! If you set a passphrase you will need to type that.<br />
<br />
<h3>
Configure the server</h3>
The key requirement for the server configuration is that by default password authentication is turned off. Find the line in <span style="font-family: Courier New, Courier, monospace;">/etc/ssh/sshd_config</span> which sets password authentication. Make sure it is set to<br />
<blockquote class="tr_bq">
<span style="font-family: Courier New, Courier, monospace;">PasswordAuthentication no</span></blockquote>
If necessary remove any leading <span style="font-family: Courier New, Courier, monospace;">#</span> character. Whilst editting <span style="font-family: Courier New, Courier, monospace;">/etc/ssh/sshd_config</span> its a good idea to disable empty passwords and root logins, ensure the following lines are set<br />
<blockquote class="tr_bq">
<span style="font-family: Courier New, Courier, monospace;">PermitEmptyPasswords no<br />
PermitRootLogin no</span></blockquote>
If you need empty passwords or root logins from specific hosts this can be overridden later.<br />
<br />
<br />
<h3>
Enable password authentication for trusted hosts</h3>
Once password authentication is disabled users on a multi-user system will have a problems copying their keys to grant them access! Exceptions can be made for trusted static IP addresses so that users can copy their public key. Assume we trust all IP addresses in the subnet 192.168.1.0/24. Passwords can then be enabled by simply adding the following lines to the end of <span style="font-family: Courier New, Courier, monospace;">/etc/ssh/sshd_config</span><br />
<blockquote class="tr_bq">
<span style="font-family: Courier New, Courier, monospace;">Match Address 192.168.1.0/24<br />
PasswordAuthentication yes</span></blockquote>
<br />
<h3>
Enable root logins</h3>
Suppose a root login using password authentication is required for one host only (192.168.1.123). Add the following lines to <span style="font-family: Courier New, Courier, monospace;">/etc/ssh/sshd_config</span><br />
<blockquote class="tr_bq">
<span style="font-family: Courier New, Courier, monospace;">Match Address 192.168.1.123<br />
PasswordAuthentication yes<br />
PermitRootLogin yes</span></blockquote>
<br />
Summary<br />
<br />
These simple steps keep your SSH server accessible from everywhere but greatly reduce the likelihood of a successful brute force attack. The Match keyword requires openssh version 4.4 or later.<br />
<br />Steve Marplehttp://www.blogger.com/profile/15582257560382364498noreply@blogger.com0tag:blogger.com,1999:blog-252642453295590453.post-26407520520179626612013-05-30T23:28:00.000+01:002013-05-30T23:28:36.414+01:00Cloud detector: first data<div class="separator" style="clear: both; text-align: center;">
</div>
<table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto; text-align: center;"><tbody>
<tr><td style="text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhjT4DzHHyUPv1LHoK0FA0d3MHSgUWdu1Kha14S1IlFKriv2pOBNTOegPiBgkS7uz4M9bDJeRdYdJrBYHGJr8Od4sYOerWOwD1znn0T7UK2p3E4hOhFg9UA6lwFm6BQHQV_CicY9XDo6oQ/s1600/today_annotated2.png" imageanchor="1" style="margin-left: auto; margin-right: auto;"><img border="0" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhjT4DzHHyUPv1LHoK0FA0d3MHSgUWdu1Kha14S1IlFKriv2pOBNTOegPiBgkS7uz4M9bDJeRdYdJrBYHGJr8Od4sYOerWOwD1znn0T7UK2p3E4hOhFg9UA6lwFm6BQHQV_CicY9XDo6oQ/s1600/today_annotated2.png" /></a></td></tr>
<tr><td class="tr-caption" style="text-align: center;">Annotated plot for 2013-05-30.</td></tr>
</tbody></table>
The cloud detector has been operating outside for a few days now. The plot above is an example of the data I am getting, annotations added manually! The <i>ambient temperature</i> is the temperature of the sensor. In reality this may be a little warmer than ambient when it is exposed to direct sunlight. The <i>sky temperature</i> is the <i>object temperature</i> from the sensor. It is an average over its field of view, and in the absence of clouds it is also averaged in height. From the plot a number of features can be identified:<br />
<br />
<b>Clear sky</b><br />
With no clouds to reflect back thermal radiation the sky temperature for a clear sky is very low, typically < -10°C. This results in a large difference (> 20°C) between ambient and sky temperatures.<br />
<br />
<br />
<b>Overcast sky</b><br />
With the presence of clouds most thermal radiation is not lost to space but reflected back to Earth and the sensor. The difference between ambient and sky temperatures is small (< 10°C). With a uniform cloud coverage the variation in the difference is small.<br />
<br />
<br />
<b>Cloudy sky</b><br />
A sky with partial cloud cover has an intermediate difference in ambient and sky temperatures. The cumulus clouds resulted in a large variation of the temperature difference, caused by the more sensitive central viewing area observing alternately clear sky and cloud. Complete coverage by a thin layer of cloud is expected to give an intermediate temperature difference but low variability.<br />
<br />
<br />
<b>Wet sensor</b><br />
When the sensor is wet with rain the sky temperature is measured incorrectly since the water is not transparent to long-wave infra-red radiation and matches the ambient temperature. As wet weather implies cloud this is not a major problem although I have concerns over the time taken for the sensor to dry off.<br />
<br />
<br />
<b>Dew</b><br />
The formation of dew is observed in the evening as the ambient temperature falls. The difference between ambient and sky temperature is reduced to less than 10°C.<br />
<br />
<h3>
Future work</h3>
Early operation shows that dew is a problem and can indicate cloud when the sky is actually clear. I have considered creative solutions such as painting the top of the box white and the bottom black in order to minimise heat lost to the sky and maximise heat absorbed from the ground. I expect this will only delay the onset when dew becomes a problem, not eliminate it. The only viable solution is to heat the sensor slightly (~2°C) above ambient, which means moving away from battery-powered operation. If a cable is used for power then wireless communications makes little sense, power-over-ethernet seems the way forward. A heated sensor will also reduce the drying time after rainfall.<br />
<br />
I still have some concerns regarding waterproofing around the sensor. I have ordered the MLX90614ESF-DCH-000-TU-ND which features a 12° field of view. This narrower view is obtained by fitting a refractive germanium lens above the sensor. The TO-39 package has been modified to be taller than standard to accommodate the lens. The additional height should mean that the top of the sensor is above the cable gland. Water should not collect so easily and it should be possible to use silicone sealant without accidentally coating the sensor window.<br />
<br />
Previously I tried pointing the sensor downwards and using a metal surface as a mirror to view the sky. The experiment was not successful and I think this was in part due to the wide field of view of the sensor (90°) preventing a clear upward view without also observing the ground or enclosure. It will be worthwhile repeating the experiment with the narrow field of view sensor.<br />
<br />
I only have basic plotting routines at present. The goal is to produce a cloud cover index, varying between 0 indicating no cloud and 1 indicating complete cloud cover. The index will be used as the basis for sending alerts.<br />
<br />
<br />Steve Marplehttp://www.blogger.com/profile/15582257560382364498noreply@blogger.com1tag:blogger.com,1999:blog-252642453295590453.post-2868148246126841392013-05-20T23:19:00.001+01:002013-06-06T20:50:40.268+01:00Cloud detector progress<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEg02ihAMqYtHZhIuQga7BCxx-1aM3iH3e9EpWbhBU8dePWn0RvsqIhfp4LqwnILlwu0Wbhj3af_MhZvzjmGU8cWc61GJTytXZduSwLWfRflfvus8gPBXacnsgT_kw6Gpq4YIensMhbhmDU/s1600/IMAG0218.jpg" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="266" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEg02ihAMqYtHZhIuQga7BCxx-1aM3iH3e9EpWbhBU8dePWn0RvsqIhfp4LqwnILlwu0Wbhj3af_MhZvzjmGU8cWc61GJTytXZduSwLWfRflfvus8gPBXacnsgT_kw6Gpq4YIensMhbhmDU/s400/IMAG0218.jpg" width="400" /></a></div>
<h2>
</h2>
<h2>
Microcontroller and radio communications</h2>
<div>
To minimise the time and effort required to test the <a href="http://blog.stevemarple.co.uk/2013/05/an-open-access-cloud-detection-network.html" target="_blank">cloud detector</a> concept I am following the approach used for the <a href="http://aurorawatch.net/" target="_blank">AuroraWatchNet</a> magnetometer system and reusing as much hardware from it as possible. The MLX90614 non-contact infra-red temperature sensor is located outside. To minimise the infrastructure requirements the sensor unit is battery-powered and transmits its data over a bi-directional radio link. The sensor is controlled by one of my <a href="http://blog.stevemarple.co.uk/2012/12/calunium-version-2.html" target="_blank">Calunium v2</a> microcontroller development boards. I'm using a pair of RFM12B radio modules operating at 433MHz to emulate a <a href="http://blog.stevemarple.co.uk/2013/01/transparent-serial-emulation-for-rfm12b.html" target="_blank">transparent serial connection</a>. One is fitted on the Calunium board and the other radio module is connected to a Raspberry Pi via my <a href="http://blog.stevemarple.co.uk/2013/02/rfm12b-shield-for-raspberry-pi.html" target="_blank">RFM12B shield</a>. The Pi records the data and is responsible for uploading it for general access. I am also reusing most of the AuroraWatchNet <a href="https://github.com/stevemarple/AuroraWatchNet" target="_blank">firmware</a>, which means I already have signed data communications and the capability to deploy over-the-air firmware updates - quite a good starting position for a new project!</div>
<div>
<br /></div>
<h2>
Waterproofing the sensor</h2>
<div>
The biggest challenge I expect to face with this project is waterproofing the sensor. Although it is hermetically sealed there is no easy way to deploy it outside. As explained in a <a href="http://blog.stevemarple.co.uk/2013/05/an-open-access-cloud-detection-network.html" target="_blank">previous post</a>, only a few materials are transparent to long-wave infra-red emissions and the better ones are toxic, water-soluble or expensive. I am therefore trying to avoid covering the sensor window and instead keep the electrical contacts dry. My first attempt was to drill a hole in the box and attach the sensor with silicone sealant. The mechanical fixing lacked strength and I wasn't convinced I had completely sealed around the sensor. To make matters worse I ended up with sealant on the sensor window which had to be cleaned off before it set. Another approach was required.</div>
<div>
<br /></div>
<div>
The new approach is to fit the sensor inside a cable gland. Before doing so I had to attach wires to the sensor connectors and insulate with heat-shrink sleeving. The result is shown below.</div>
<div>
<br /></div>
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhJG0RVwEVO8vymPQGf2ehDyRaEM28dtlgpW8T8clFgJOpcqcEtx-8CtHhpwPpkO0Uva-7C8D9vpCqYNXbBsPlGGjY-stEQnPzeq6YKx_TPHJUjPcKGQpaUowVQYC007MdTtq9TyMBEBjk/s1600/IMAG0229.jpg" imageanchor="1"><img border="0" height="213" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhJG0RVwEVO8vymPQGf2ehDyRaEM28dtlgpW8T8clFgJOpcqcEtx-8CtHhpwPpkO0Uva-7C8D9vpCqYNXbBsPlGGjY-stEQnPzeq6YKx_TPHJUjPcKGQpaUowVQYC007MdTtq9TyMBEBjk/s320/IMAG0229.jpg" width="320" /></a></div>
<div class="separator" style="clear: both; text-align: center;">
<br /></div>
<br />
The next step is to fit the sensor in the cable gland before fitting the cable gland to the box.<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgHO_GZpYHeS8Y3JGSvpG7wn_zETleTuiDbaigyFBL9asj8kyRHfxz1d7BI3Eo1vmyF_r7FZXUzFWL-MBCO70pmf8ENo9xb7No9tEzwJ9neqFh_jXet8ue1SjIporqTMModnTw73ZeTd4g/s1600/IMAG0235.jpg" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="320" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgHO_GZpYHeS8Y3JGSvpG7wn_zETleTuiDbaigyFBL9asj8kyRHfxz1d7BI3Eo1vmyF_r7FZXUzFWL-MBCO70pmf8ENo9xb7No9tEzwJ9neqFh_jXet8ue1SjIporqTMModnTw73ZeTd4g/s320/IMAG0235.jpg" width="213" /></a></div>
<div>
<br /></div>
<div>
<br /></div>
<div>
The other end of the wires can then be soldered. For the prototype the wires are soldered to a small piece of stripboard. The stripboard has two pull-up resistors for the I2C bus and a decoupling capacitor. There is also an IDC connector to link the board to the unused JTAG interface. This was the easiest way to connect to a stock Calunium v2 board. For the lowest power operation the sensor is powered from logic-level output and I am using software I2C so all connections can be grounded when the sensor is not in use. The real-time clock on Calunium is connected the the hardware I2C interface on the microntroller and can be operated independently of the sensor.</div>
<div>
<br /></div>
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEh9SQfyA2L0_6wbdbfF3E6OBKBnfvRqzOEUP4cZ1SpksWgZMdkRn0UOvDSBergtyYZWtJzY7wf5_o81-KXUhPfTZ2FbiaB-mCMbUsmF-gj4IJEGJrh4K_O0awNWFXfEdf8IxoWkRXy35BQ/s1600/IMAG0234.jpg" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="213" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEh9SQfyA2L0_6wbdbfF3E6OBKBnfvRqzOEUP4cZ1SpksWgZMdkRn0UOvDSBergtyYZWtJzY7wf5_o81-KXUhPfTZ2FbiaB-mCMbUsmF-gj4IJEGJrh4K_O0awNWFXfEdf8IxoWkRXy35BQ/s320/IMAG0234.jpg" width="320" /></a></div>
<div>
<br /></div>
<div>
<br /></div>
<div>
The box and cable gland are rated <a href="http://en.wikipedia.org/wiki/IP_Code" target="_blank">IP65</a>. Hopefully the result will be waterproof </div>
<div>
<br /></div>
<h2>
Results</h2>
<div>
The sensor is now deployed outside and reporting back to the Raspberry Pi base station. The transmitted packets are logged by the Raspberry Pi but as yet I don't have a convenient way to extract the ambient and object temperatures for analysis. That is the next task.</div>
<div>
<br /></div>
<div>
<br /></div>
Steve Marplehttp://www.blogger.com/profile/15582257560382364498noreply@blogger.com0tag:blogger.com,1999:blog-252642453295590453.post-34359324423179195262013-05-18T11:58:00.000+01:002014-04-18T14:59:55.508+01:00An open-access cloud-detection network<div dir="ltr" style="text-align: left;" trbidi="on">
<h2>
Introduction</h2>
Living in the North-West of England I find that my attempts to view the aurora or astronomical events such as meteor showers, lunar eclipses and comets are often prevented by cloud. On these occasions I wonder if it is worthwhile driving somewhere to find some clear sky. But where? If only I could look at a map to see where it was clear now. Publicly-available satellite images don't appear to have the resolution needed for this purpose. Maybe I should go to bed instead and wait for an alert when the skies have cleared? Could the question of where and when to find clear skies be answered by an open-access cloud detection network operated by citizen-science volunteers?<br />
<br />
<h2>
Requirements</h2>
I've together a draft list of requirements to construct a useful network:<br />
<br />
<ul>
<li>Open-access. While my primary concern is with viewing the aurora I can see there are many other uses of such a network. Astronomy and obviously meteorology. Open-access enables other observers to make use of the data for their own purposes.</li>
<li>Near real-time. The weather here can quickly change. Ideally measurements should be taken every few minutes and be made available without unnecessary delay.</li>
<li>Use open-source hardware (<a href="http://freedomdefined.org/OSHW" target="_blank">OSHW</a>). More specifically, an OSHW design should exist which allows users to contribute readings with the lowest possible cost.</li>
<li>Large geographical coverage. </li>
<li>Night time operation required, day time beneficial.</li>
</ul>
<div>
The most difficult requirement to satisfy is the coverage. With ground-based sensors the viewing area might be as small as 2km diameter (cloud-base 1000m, optical viewing angle ±45°); at this size complete coverage of Great Britain would require over 57000 installations. A useful network does not require complete coverage, for auroral and astronomical purposes good results could be achieved by positioning sensors in a few viewing locations selected for low light pollution.</div>
<div>
<br /></div>
<h2>
Remotely-sensing clouds</h2>
Perhaps the simplest method to remotely-sense cloud cover is to measure the 'sky' temperature with a non-contact infra-red (IR) thermometer. In the absence of cloud cover this is very low (say -12 to -20°C for Spring in the UK) but with cloud cover the temperature measured is warmer (0 to 4°C). These temperatures vary over the course of the year but by looking at the difference between the sky and ambient ground-level temperatures an estimate of cloud cover is possible. Some of these <a href="http://en.wikipedia.org/wiki/Thermopile" target="_blank">thermopile</a> sensors have a digital output for interfacing directly with a microcontroller.<br />
<br />
A camera and image-recognition software is another approach but it is more complex to develop, install and operate. It is not clear how such a system would perform at night. In order to install the maximum number of sensors I plan to concentrate on cheaper methods.<br />
<br />
<h2>
Design concept</h2>
I believe there are two key factors which limit the number of sensors in the network: cost and ease of installation. To minimise the infrastructure I plan to begin with a battery-powered unit which communicates with a Raspberry Pi (or other computer) via a low-cost, licence-free radio module. This is the same concept used by the <a href="http://blog.stevemarple.co.uk/search/label/AuroraWatchNet" target="_blank">AuroraWatchNet</a> magnetometer sensors.<br />
<br />
<br />
<h2>
Potential problems</h2>
<div>
Precipitation (rain, snow, hail) and dew on the sensor will prevent accurate sky temperature measurements, most likely indicating cloud when it may in fact be clear. A heater to prevent dew forming is an option but the temperature gradients may cause the thermopile to give incorrect measurements. The power consumption needed is incompatible with battery operation. It <b>may</b> be possible to point the sensor downwards, to avoid precipitation and dew, and use a reflective metal surface to observe the sky. Tests shows the sky temperature readings are partially compromised by this approach.</div>
<br />
<br />
<h2>
Sensor</h2>
I have found a number of sensors which may be suitable:<br />
<ul>
<li>Melexis MLX90614. This sensor contains thermopiles for sensing both the object and ambient temperature. An SMBus interface enables direct connection to a microcontroller. It is packaged in a TO-39 metal can. The standard 3.3V operation, 90° field of view (FOV) variant is readily available in single quantities from UK hobby electronics stores for around £15 each. Other variants exist for 5V operation and with different fields of view (10°, 35°, and a dual 70° FOV). The minimum object temperature is -70°C. The only irritation is that the sleep mode requirements are not compatible with low-power operation on a I2C bus.</li>
<li>Melexis MLX90615. Similar to the MLX90614 but the sleep mode requirements are compatible with low-power operation on an I2C bus. Cost is similar to MLX90614 but no UK supplier exists. Package is a TO-46 metal can. Limited to 3.3V operation and a single FOV (80° or 100° variants). The minimum object temperature is -40°C.</li>
<li>Texas Instruments TMP006. Small thermopile with I2C interface for direct connection to a microcontroller. Ball-grid array package. Readily available from UK suppliers. Cost is around £3 each for single quantities.</li>
<li>Thermometrics ZTP-135SR. Thermopile with analogue output in a TO-46 package, with a thermistor for ambient temperature measurement. Being an analogue output additional processing is required. Around £10 from Farnell.</li>
<li>Devantech TPA81. Linear array of 8 thermopiles. Cost is around £65.</li>
<li>Panasonic AMG88 Grid-EYE. An 8×8 imaging sensor in a surface-mount package. The minimum object temperature is -40°C. Available from Digikey for around $40.</li>
</ul>
<div>
For the prototype system I plan to use the Melexis MLX90614 sensor. It should be possible to work around the problems with low-power operation by using software I2C and powering the sensor from a logic output, allowing power to be completely removed when not in use.<br />
<br />
I excluded the TMP006 because the BGA package is hard to work with; not only is it difficult to solder it will require some kind of enclosure or window to protect it from moisture. Only a few materials are transparent to long-wave infra-red emissions and the better ones are toxic, water-soluble or expensive. The other sensors may be suitable but are more expensive. The TPA-81 linear array and Grid-EYE imaging sensor could provide spatial discrimination and thus distinguish between a thin layer of complete cloud cover and denser clouds with open areas of sky.</div>
<div>
<br />
<h2>
Initial results</h2>
</div>
<div>
Initial tests with the Melexis MLX90614 sensor confirm the concept is viable. I have not noticed any ill-effects from the sun within the field of view, suggesting that daytime operation is possible. Further work is required to package the system into a usable prototype which can be deployed outside permanently. Full-time operation will indicate the frequency of measurement errors from dew and precipitation.<br />
<br /></div>
<h2>
Other work</h2>
<div>
It would be wrong to suggest I am the first to make measurements of cloud cover in this way, useful links are listed below. I am not aware of any existing cloud detection networks.</div>
<div>
<br /></div>
<div>
<a href="http://bildr.org/2011/02/mlx90614-arduino/">http://bildr.org/2011/02/mlx90614-arduino/</a></div>
<div>
Useful code for interfacing the MLX90614 IR thermometer with an Arduino. This code works with my <a href="http://blog.stevemarple.co.uk/search/label/Calunium" target="_blank">Calunium</a> Arduino clone.</div>
<div>
<br /></div>
<div>
<a href="http://www.kwos.org/clouds_detector.htm">http://www.kwos.org/clouds_detector.htm</a></div>
<div>
Good background information.</div>
<div>
<br />
<a href="http://kcotar.org/arduino-weather-station-1/">http://kcotar.org/arduino-weather-station-1/</a><br />
A weather station which uses the Melexis MLX90614 sensor for cloud detection.<br />
<br /></div>
<a href="http://www.noao.edu/staff/gillespie/projects/cloud-detector.html">http://www.noao.edu/staff/gillespie/projects/cloud-detector.html</a><br />
Similar in concept but using a Peltier device (in reverse) to measure the difference between ambient ground temperature and sky temperature.<br />
<br />
<br /></div>
Steve Marplehttp://www.blogger.com/profile/15582257560382364498noreply@blogger.com4tag:blogger.com,1999:blog-252642453295590453.post-74150525207538954462013-03-12T00:12:00.001+00:002013-03-12T00:23:57.937+00:00How to use the GPIO version of avrdude on the Raspberry Pi<h3>Introduction</h3>I previously showed an implementation of a <a href="http://blog.stevemarple.co.uk/2012/07/avrarduino-isp-programmer-using.html" target="_blank">AVR ISP programmer using the Raspberry Pi GPIO port</a> which can be used to program Atmel's AVR range of microcontrollers with avrdude. An ISP programmer based on this design was incorporated into a <a href="http://blog.stevemarple.co.uk/2013/02/rfm12b-shield-for-raspberry-pi.html" target="_blank">shield to interface to the RFM12B radio module</a>. This post explains how to use <code>avrdude</code> to actually program devices.<br />
<br />
<h3>Software installation</h3>Download the patched version of avrdude from <a href="http://project-downloads.drogon.net/files/">http://project-downloads.drogon.net/files/</a>. I also keep a copy in the <code>RPi_RFM12B_ISP</code> respository, <a href="https://github.com/stevemarple/RPi_RFM12B_ISP/tree/master/software/avrdude">https://github.com/stevemarple/RPi_RFM12B_ISP/tree/master/software/avrdude</a>. You will probably want the armhf (hardware floating-point) version. Download the documentation package for avrdude too. Install the packages using "dpkg -i". For example<br />
<br />
<pre>sudo dpkg -i avrdude_5.10-4_armhf.deb
sudo dpkg -i avrdude-doc_5.10-4_all.deb</pre><br />
Using avrdude over the GPIO interface is problematic for users other than root. The easiest solution is to give the avrdude binary setgroup permission:<br />
<pre>sudo chmod g+s /usr/bin/avrdude</pre><br />
<h3>Usage</h3>Selecting the GPIO programmer is simply a matter of including "<code>-P gpio -c gpio</code>" options; the <code>-P</code> option specifies that the GPIO port is used (as opposed to USB, serial or parallel interfaces) whilst the <code>-c</code> option selects the correct programmer type on that port.<br />
<br />
For example, to check the signature on an ATmega328P execute the command<br />
<pre>avrdude -P gpio -c gpio -p atmega328p</pre><br />
To read the fuses execute the command<br />
<pre>avrdude -P gpio -c gpio -p atmega328 -U lfuse:r:-:h -U hfuse:r:-:h -U efuse:r:-:h</pre><br />
<h3>Customisation</h3>The packages above define a single programmer called <code>gpio</code> which uses the <code>gpio</code> interface on GPIO pins 8 to 11. Since the <a href="http://blog.stevemarple.co.uk/2013/02/rfm12b-shield-for-raspberry-pi.html" target="_blank">RFM12B shield for Raspberry Pi</a> implements two independent programmers I prefer to use <code>gpio0</code> and <code>gpio1</code>. You can add these by creating a <code>.avrduderc</code> file in your home directory. The file should contain:<br />
<br />
<pre>programmer
id = "gpio0";
desc = "Use sysfs interface to bitbang GPIO lines";
type = gpio;
reset = 8;
sck = 11;
mosi = 10;
miso = 9;
;
programmer
id = "gpio1";
desc = "Use sysfs interface to bitbang GPIO lines";
type = gpio;
reset = 7;
sck = 11;
mosi = 10;
miso = 9;
;
</pre><br />
Steve Marplehttp://www.blogger.com/profile/15582257560382364498noreply@blogger.com38tag:blogger.com,1999:blog-252642453295590453.post-76456928046627595122013-03-03T09:17:00.000+00:002013-04-20T15:54:26.689+01:00Three-axis sensor unit for AuroraWatchNet<table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto; text-align: center;"><tbody>
<tr><td style="text-align: center;"><a href="http://www.flickr.com/photos/stevemarple/8517329264/" style="margin-left: auto; margin-right: auto;" title="Three-axis magnetometer sensor by Steve Marple, on Flickr"><img alt="Three-axis magnetometer sensor unit" height="500" src="http://farm9.staticflickr.com/8093/8517329264_8796f75a7e.jpg" width="500" /></a></td></tr>
<tr><td class="tr-caption" style="text-align: center;">Three-axis sensor unit. The RJ-45 connector is for power and I2C data signals, not Ethernet.</td></tr>
</tbody></table>
<br />
<a href="http://aurorawatch.net/" target="_blank">AuroraWatchNet</a> is a network of magnetometers for auroral alerts and citizen science. Whilst the magnetometer is intended to function with one single-axis <a href="http://www.stefan-mayer.com/Lcsing.htm" target="_blank">FLC100</a> sensor from <a href="http://www.stefan-mayer.com/" target="_blank">Stefan-Mayer Instruments</a> I have designed the system to support 3-axis operation by mounting 3 sensor boards at 90° to each other using just standard right-angle PCB connectors. This approach avoids the requirement for external mounting hardware which would be expensive to manufacture.<br />
<br />
The power supply (a charge-pump DC-DC converter), temperature sensor and RJ-45 connector are fitted to just one circuit board. It is possible to use only one analogue-to-digital converter (ADC) but given their low cost it was presumed they would be fitted to all boards to enable simultaneous sampling of all 3 axes. The MCP3424 converter was chosen because eight options are available for the I2C address, thus jumpers can be fitted to select a unique address for every ADC on the I2C bus. This converter also features 4 input channels. All magnetometer outputs and the temperature sensor are connected to each ADC so that the spare channels can provide some redundancy in case of an ADC failure. The SYNC connectors of all sensors are linked to synchronise their excitation frequencies.<br />
<br />
Several single-axis units have been built and one is in active use by AuroraWatch UK. The photograph above shows the first three-axis unit to be built. Initial testing indicates it functions as intended but the performance of 3 simultaneously-powered sensors in close proximity is yet to be established.<br />
<div>
<br /></div>
<br />Steve Marplehttp://www.blogger.com/profile/15582257560382364498noreply@blogger.com6tag:blogger.com,1999:blog-252642453295590453.post-90428112842103733382013-02-10T00:02:00.000+00:002013-04-20T15:54:49.499+01:00The importance of brown-out detectionThe batteries expired in one of the <a href="http://aurorawatch.net/" target="_blank">AuroraWatchNet</a> test magnetometers this week. I was expecting that to happen since the magnetometer wasn't operating in its low-power mode and the battery voltage from the two 1.5V alkaline cells had been hovering around 2V for some time. The Atmel ATmega1284P microcontroller is actually powered at 3.3V using a boost power supply. What I didn't expect was for the system to not start after replacing the battery. I could tell <a href="http://blog.stevemarple.co.uk/2012/11/using-xboot-with-calunium.html" target="_blank">xboot</a> was working since the LED flashed 3 times as usual after the reset button was pressed. After that the LED stayed on but nothing else happened, not even the start-up debug messages from the serial port. I guessed the flash program memory had somehow got corrupted.<br />
<br />
I used the AVR Dragon to extract the contents of the flash memory (<code>avrdude -P usb -c dragon_jtag -p atmega1284p -U flash:r:flash.img:r</code>) and compared it against the saved raw image file using the linux <code>split</code> and <code>cmp</code> commands. Sure enough, three pages of flash memory were set to <code>0xFF</code>, probably from some erroneous page clear operation. Brown-out detection was disabled by the fuse settings to save power. Big mistake! What I think happened was as the voltage dipped below its valid range the microcontroller started executing random commands, including 3 SPM page clears, one of which was the first page of flash which holds the interrupt vector table. Valid execution was doomed.<br />
<br />
What I should have done was to enable brown-out detection in the fuses, 2.7V seems the most appropriate setting for an 8MHz clock and 3.3V operation. To save power during sleeping the brown-out detection can be disabled with a call to <code><a href="http://www.nongnu.org/avr-libc/user-manual/group__avr__sleep.html" target="_blank">sleep_bod_disable()</a></code>. Lesson learned!Steve Marplehttp://www.blogger.com/profile/15582257560382364498noreply@blogger.com1tag:blogger.com,1999:blog-252642453295590453.post-88768772291980138282013-02-09T13:02:00.001+00:002013-02-19T23:17:53.762+00:00Calunium: version 2.1<table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto; text-align: center;"><tbody>
<tr><td style="text-align: center;"><a href="http://www.flickr.com/photos/stevemarple/8458537372/" style="margin-left: auto; margin-right: auto;" title="Calunium v2.1 by Steve Marple, on Flickr"><img alt="Calunium v2.1" src="http://farm9.staticflickr.com/8530/8458537372_1ab93377b2.jpg" /></a></td></tr>
<tr><td class="tr-caption" style="text-align: center;">Calunium v2.1.<br />
Click on the image for an annotated version.</td></tr>
</tbody></table>
<br />
This upgrade to <a href="http://blog.stevemarple.co.uk/2012/12/calunium-version-2.html" target="_blank">Calunium v2</a> provides several minor improvements. There is the option to fit a <a href="http://blog.stevemarple.co.uk/2013/01/uart-usb-converter-for-mixed-33v5v.html" target="_blank">Microchip MCP220 UART-USB converter</a> instead of the FTDI connector. For development purposes the MCP2200 is much more convenient than an external FTDI cable or adaptor; operation at both 3.3V and 5V is supported. For low-power battery operation the MCP2200 uses too much power so the FTDI connector option has been retained.<br />
<br />
I find I operate Calunium almost exclusively at 3.3V. I've not been able to find any real-time clocks in a dual-in-line package that operate at 3.3V and feature battery-backup. Suitable clocks are supplied in a SOIC or smaller package. I have therefore removed the DIP footprint option for the real-time clock. The space saved has allowed me to add optional load capacitors for the crystal so that a MicroChip MCP7941x real-time clock can be used. This clock has several benefits over the popular DS1307 (or its 3.3V equivalent, the DS1338). It is cheaper, it can compensate for crystal frequency errors, it has EEPROM storage, two alarms and it also has a unique identifier. This change goes against one of the original aims of Calunium, to use through-hole components to keep it accessible to the widest possible audience. However, considering the other interesting on-board peripherals (micro-SD socket, RFM12B and MCP2200) require surface-mount soldering skills I thought this a reasonable compromise to make. The original through-hole only designs are still available for those who really do not want to use any surface-mount components.<br />
<br />
The final improvement is that the SPI clock signal is now routed with a ground plane between it and the real-time clock crystal.<br />
<br />
For the full list of features please see the <a href="http://blog.stevemarple.co.uk/2012/12/calunium-version-2.html" target="_blank">original Calunium v2 blog post</a>.<br />
<br />
<table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto; text-align: center;"><tbody>
<tr><td style="text-align: center;"><a href="http://www.flickr.com/photos/stevemarple/8458540722/" style="margin-left: auto; margin-right: auto;" title="Calunium v2 and v2.1 PCBs by Steve Marple, on Flickr"><img alt="Calunium v2 and v2.1 PCBs" src="http://farm9.staticflickr.com/8102/8458540722_bf0aa314d2.jpg" /></a></td></tr>
<tr><td class="tr-caption" style="text-align: center;">Calunium v2 (top) and v2.1 (bottom) PCBs.<br />
Click on the image for an annotated version.</td></tr>
</tbody></table>
Steve Marplehttp://www.blogger.com/profile/15582257560382364498noreply@blogger.com26tag:blogger.com,1999:blog-252642453295590453.post-83606145264515218072013-02-08T21:40:00.000+00:002013-03-13T15:55:05.522+00:00RFM12B shield for Raspberry Pi<table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto; text-align: center;"><tbody>
<tr><td style="text-align: center;"><a href="http://www.flickr.com/photos/stevemarple/8452549059/" style="margin-left: auto; margin-right: auto;" title="RFM12B and Atmel ISP programmer shield for Raspberry Pi by Steve Marple, on Flickr"><img alt="RFM12B and Atmel ISP programmer shield for Raspberry Pi" height="333" src="http://farm9.staticflickr.com/8378/8452549059_d1e8b22817.jpg" width="500" /></a></td></tr>
<tr><td class="tr-caption" style="text-align: center;">RFM12B shield for Raspberry Pi, with integrated in-circuit serial programmer.<br />
Click on the image for an annotated version. </td></tr>
</tbody></table><br />
I recently described the <a href="http://blog.stevemarple.co.uk/2012/12/rfm12b-shield-for-raspberry-pi.html" target="_blank">preliminary design</a> of a RFM12B shield for the Raspberry Pi, which used an Atmel ATmega328 to interface between the Raspberry Pi's UART and the RFM12B's SPI interface. The design also included a buffered two-channel in-circuit serial programmer (ISP) using the Raspberry Pi's general purpose I/O. The first channel is for programming the on-board ATmega328 whilst the second channel can program external microcontrollers.<br />
<br />
The boards have now been manufactured and the first one assembled and tested. The interface to the RFM12B and the built-in AVR ISP programmer both work as designed. I have been using the Arduino environment to compile and upload the firmware to the ATmega328. The microcontroller pin mapping is identical to the Arduino Uno although the activity LED is wired to D9 since D13 is normally in use by the SPI interface. The ATmega328 operates at 3.3V and is clocked by the internal 8MHz RC oscillator. No bootloader is needed. The <a href="https://github.com/stevemarple/RPi_RFM12B_ISP" target="_blank">Github</a> repository contains <code>boards.txt</code> and <code>programmers.txt</code> files for use with the Arduino IDE, the standard Arduino core is used.<br />
<br />
I have been using the first board to replace a <a href="http://shop.ciseco.co.uk/urf-radio-module-and-serial-inteface-via-usb/" target="_blank">Ciseco URF</a> radio module. The firmware for the ATmega328 incorporates my <a href="http://blog.stevemarple.co.uk/2013/01/transparent-serial-emulation-for-rfm12b.html" target="_blank">transparent serial emulation layer</a>. This provides a <code>Stream</code> object to which data can be read, and written (or printed), in just the same way as <code>HardwareSerial</code> and <code>SoftwareSerial</code> objects. The same emulation layer can then be used on remote <a href="http://blog.stevemarple.co.uk/search/label/Calunium" target="_blank">Calunium</a>, Arduino or JeeNode boards to provide a bi-directional data stream, allowing the user to interface to the RFM12B as easily as an XRF or XBee radio module. The emulation layer transparently uses acknowledgements and retries to provide a robust channel, which is an improvement over the Ciseco XRF and URF radio modules. Alternative firmware for the RFM12B shield is easily uploaded to provide different functionality.<br />
<br />
<h3>ISP programmer</h3>The avrdude configuration file contains details for two GPIO programmers; <code>gpio0</code> is connected to the on-board ATmega328 and <code>gpio1</code> is for programming external microcontrollers. By fitting a jumper to the JP1 header the external microcontroller can be powered from the Raspberry Pi's 3.3V supply. Be sure to observe the maximum current rating for the 3.3V supply and never fit the jumper if the remote microcontroller is self-powered. The 74LVC244 buffer ensures that MCUs operating at 5V can be safely connected. Typically the high output level from the 74LVC244 buffer will be recognised as a high but it doesn't meet the worst-case specifications when the external MCU is powered from 5V.<br />
<br />
<h3>Open source</h3>The Eagle PCB design files for the RFM12B/ISP shield for Raspberry Pi are available on <a href="https://github.com/stevemarple/RPi_RFM12B_ISP" target="_blank">Github</a> and are licensed under the <a href="http://creativecommons.org/licenses/by-sa/3.0/" target="_blank">Creative Commons Attribution-ShareAlike 3.0 Unported License</a>. The RF12 transparent serial emulation layer is also available on <a href="https://github.com/stevemarple/RF12_Stream" target="_blank">Github</a> and is released under the <a href="http://opensource.org/licenses/mit-license.php" target="_blank">MIT license</a>.<br />
<br />
<h3>Errata</h3>There is a connection missing on the schematic from the drain of FET Q2 to pin 5 (/RESET) on the external ISP header (X3). This connection is also missing from the PCB but can be easily fixed with a direct wire connection on the underside of the board.<br />
<br />
<h3>Update</h3>I've added a description of how to install and use the patched version of avrdude, <a href="http://blog.stevemarple.co.uk/2013/03/how-to-use-gpio-version-of-avrdude-on.html">http://blog.stevemarple.co.uk/2013/03/how-to-use-gpio-version-of-avrdude-on.html</a>.Steve Marplehttp://www.blogger.com/profile/15582257560382364498noreply@blogger.com17tag:blogger.com,1999:blog-252642453295590453.post-30729548595785402832013-01-30T11:08:00.000+00:002013-02-08T10:39:37.096+00:00Transparent serial emulation for the RFM12B radio module<h3>Introduction</h3>For the <a href="http://aurorawatch.net/" target="_blank">AuroraWatchNet</a> <a href="http://blog.stevemarple.co.uk/search/label/Magnetometer" target="_blank">magnetometers</a> that I am developing I currently use the Ciseo <a href="http://shop.ciseco.co.uk/xrf-wireless-rf-radio-uart-rs232-serial-data-module-xbee-shape-arduino-pic-etc/" target="_blank">XRF</a> radio module. They are based upon the pseudo-standard XBee footprint, operate at 3.3V and provide a UART interface to the microcontroller. They are easy to use but I have found the range is not always sufficient, typically limited to 60 metres of free space plus one exterior wall/window. Another commonly-used radio module is the HopeRF <a href="http://www.hoperf.com/rf/fsk/21.htm" target="_blank">RFM12B</a> transceiver, as used on the <a href="http://jeelabs.net/projects/hardware/wiki/JeeNode" target="_blank">JeeNode</a>. The RFM12B also operates at 3.3V but features an SPI interface to the microcontroller. It is available as a 2mm pitch surface-mount module, or as a 12 pin 2mm pitch DIP package.<br />
<br />
As my <a href="http://blog.stevemarple.co.uk/2012/12/calunium-version-2.html" target="_blank">Calunium v2</a> microcontroller development board can be fitted with a RFM12B radio module I naturally wanted to compare the performance of the RFM12B and XRF. Initial testing of the Hope <a href="http://www.hoperf.com/rf/fsk/21.htm" target="_blank">RFM12B</a> gave a usable range of 150 metres of free space after transmission through one exterior wall/window. Some of the additional range is due to using the 433MHz version of the RFM12B compared to the 868MHz operating frequency that the XRF defaults to. The range measurements were not very scientific and further work is required before meaningful conclusions can be drawn regarding the comparative ranges. However these initial results have encouraged me to investigate replacing the XRF by the RFM12B, particularly as it is both cheaper and uses less power. A comparison of the basic specifications can be found at <a href="http://blog.homelabs.org.uk/wireless-connectivity/">http://blog.homelabs.org.uk/wireless-connectivity/</a>.<br />
<br />
As I have already deployed a few magnetometers using the Ciseco XRF module I would like to to use the RFM12B as a serial device. This would enable the existing firmware to be used with minimal changes. It would also allow the possibility of having a single firmware support both XRF and RFM12B modules.<br />
<br />
<h3>Serial emulation</h3>I have created a serial emulation layer for the RFM12B by deriving a class (<code>RF12_Stream</code>) from the Arduino <code>Stream</code> class. Text or binary data can then be read from or written to the RFM12B in exactly the same way as the standard <code>Serial</code> port. Since it is a <code>Stream</code> object the <code>Streaming</code> class (which overloads the >> and << operators) can also be used with <code>RF12_Stream</code>. The <code>RF12_Stream</code> class automatically breaks up the data to be transmitted into short packets which are sent using the Jeelab RF12 library. ACKs and retransmissions are employed to prevent losing data from dropped packets. A packet number is included to prevent duplicated data, which could otherwise occur for the case when the ACK is not received and a packet is retransmitted. The class keeps track of the number of packets sent and received, and the number of retransmissions, data which may be useful to identify the optimal channel on which to operate.<br />
<br />
The <code>begin()</code> function takes different parameters to <code>Serial.begin()</code> and it returns a logical value to indicate whether the RFM12B has been found on the system. The user is required to call the <code>poll()</code> function periodically so that outgoing packets are sent and received packets processed. These are the only differences compared to sending data to the standard Arduino <code>Serial</code> port.<br />
<br />
Source code for the <code>RF12_Serial emulation</code> can be found on Github, <a href="https://github.com/stevemarple/RF12_Stream">https://github.com/stevemarple/RF12_Stream</a>; it is released under the <a href="http://opensource.org/licenses/mit-license.php" target="_blank">MIT license</a>. An example sketch is included which echoes data between the Arduino serial port and the RFM12B. When compiled for the ATtiny84 this sketch is less than 8kB so I am hopeful it can be used on the <a href="http://shop.openenergymonitor.com/raspberry-pi/" target="_blank">RFM12B to Raspberry Pi expansion board</a> designed by <a href="http://martin%20harizanov/" target="_blank">Martin Harizanov</a>. <code>AsyncDelay</code>, <a href="https://github.com/stevemarple/AsyncDelay">https://github.com/stevemarple/AsyncDelay</a>, and <code>CircBuffer</code>, <a href="https://github.com/stevemarple/CircBuffer">https://github.com/stevemarple/CircBuffer</a>, are also required.<br />
<br />
Calunium uses <code>INT2</code> for the interrupt from the RFM12B which is not supported by the JeeLabs version of the RF12 library so I am currently using a modified version. Source for this version is also available on Github, <a href="https://github.com/stevemarple/RF12">https://github.com/stevemarple/RF12</a>. It is intended that <code>RF12_Serial</code> should work with both the original <code>RF12</code> library and my version.<br />
<br />
<h3>Single firmware to support RFM12B and XRF</h3>By utilising the return value from the <code>RF_Stream</code> <code>begin()</code> function it becomes trivial to produce a single firmware which can support either the RFM12B or XRF radio modules. The trick is to create a <code>Stream</code> reference which is set to either <code>Serial</code> (for the XRF) or to <code>rfm12b</code>. The <code>RFM12B_autoselect</code> example sketch illustrates this concept.<br />
<br />
<h3>How does it differ from <code>RF12sio</code>?</h3>The <code>RF12sio</code> library overloads the << and >> operators so that packets can be constructed and processed easily. The user is still responsible splitting the data into packets. <code>RF12sio</code> is not derived from <code>Stream</code> so it is not possible to use the <code>Stream</code> pointer trick shown in the <code>RFM12B_autoselect</code> sketch.<br />
<br />
<h3>Credits</h3><div>Thanks to <a href="http://jeelabs.net/" target="_blank">JeeLabs</a> and <a href="https://github.com/jcw/jeelib" target="_blank">JC Wippler</a> for the RF12 library and to <a href="http://arduino.cc/" target="_blank">Arduino</a> for the standard Arduino libraries.</div><div><br />
</div>Steve Marplehttp://www.blogger.com/profile/15582257560382364498noreply@blogger.com6tag:blogger.com,1999:blog-252642453295590453.post-4963872228989188602013-01-24T21:59:00.000+00:002013-01-24T21:59:45.077+00:00How to use the Atmel ATmega1284 (non-P version) with tools that don't support itI found out by accident there is now an Atmel ATmega1284 microcontroller, as well as the older ATmega1284P. The first I knew was when I tried to upload the bootloader and avrdude warned of a signature mismatch:<br />
<blockquote class="tr_bq">
<br />avrdude: Device signature = 0x1e9706<br />avrdude: Expected signature for ATMEGA1284P is 1E 97 05<br /> Double check chip, or use -F to override this check</blockquote>
<br />
Correcting the device didn't help, avrdude doesn't recognise the ATmega1284. That could be fixed easily by creating a custom entry in $HOME/.avrduderc. I didn't bother with that since avr-gcc doesn't support the ATmega1284. At this point I was slightly worried as I had ordered a large quantity of ATmega1284 instead of the ATmega1284P.<br />
<br />
<h3>
A work-around</h3>
Fortunately there is a work-around. The two microcontrollers are essentially identically except for some small electrical differences relating to power usage and the last signature byte. Therefore code compiled for the ATmega1284P will run on the ATmega1284. If you are using avrdude to upload just add the -F command line option to force the signature check to pass.<br />
<br />
If you use the <a href="http://blog.stevemarple.co.uk/2012/11/using-xboot-with-calunium.html" target="_blank">xboot</a> bootloader you should have no further problems after installing the bootloader since xboot doesn't check the signature bytes. Users of the Optiboot bootloader are probably not so fortunate; I believe the signature is checked - I haven't tested it since I'm now using xboot on all of my <a href="http://blog.stevemarple.co.uk/search/label/Calunium" target="_blank">Calunium</a> development boards.<br />
<br />Steve Marplehttp://www.blogger.com/profile/15582257560382364498noreply@blogger.com1tag:blogger.com,1999:blog-252642453295590453.post-53924206347830333052013-01-13T21:07:00.000+00:002013-04-20T15:55:06.713+01:00Magnetometer progress report<h3>
About AuroraWatchNet</h3>
<div>
I am developing a simple, low-cost, battery-powered magnetometer for auroral alerts and citizen science. It is intended to be wireless to allow easy installation, a unique feature as far as I am aware. The magnetometer uses my own <a href="http://blog.stevemarple.co.uk/2011/11/calunium-pcb-version.html" target="_blank">Calunium</a> micro-controller board, which is an <a href="http://arduino.cc/" target="_blank">Arduino</a> clone based on the ATmega1284P micrcontroller. Data is sent via a radio link to a <a href="https://www.google.co.uk/url?sa=t&rct=j&q=&esrc=s&source=web&cd=1&cad=rja&sqi=2&ved=0CEoQFjAA&url=http%3A%2F%2Fwww.raspberrypi.org%2F&ei=Tr3LUJyFIYSR0AXO6IHYBQ&usg=AFQjCNEBMoebclm0Gk0LCZIStJbF04U1cQ&sig2=8UrWPhhbq2LUuqAVlzWgSA&bvm=bv.1355325884,d.d2k" target="_blank">Raspberry Pi</a> base station which will then forward the data to AuroraWatch UK. I am hoping to deploy a network of such magnetometers in 2013 to enable AuroraWatch UK users to contribute data to improve the service. If you want to track the development then follow <a href="https://twitter.com/aurorawatchnet" target="_blank">@aurorawatchnet</a> on Twitter. </div>
<div>
<br /></div>
<div>
The first AuroraWatchNet magnetometer is currently running in my garden. The environment is not ideal as I can see disturbances from cars and even the garage door being opened but it is convenient for testing and representative of what to expect from user-contributed data. I have been comparing the results from our <a href="http://spears.lancs.ac.uk/samnet" target="_blank">SAMNET</a> magnetometers with the AuroraWatchNet prototypes and generally the results are very encouraging given the difference in cost between the two systems. Now I can plot the AuroraWatchNet magnetometer data in the standard AuroraWatch UK style, using the same <a href="http://www.gnu.org/software/octave/" target="_blank">Octave</a> code. Below is a comparison of today's plots. </div>
<div>
<h3>
AuroraWatch UK</h3>
<div>
The data is taken from SAMNET's Lancaster magnetometer. The X axis is time and the Y axis shows the magnetic field strength in the direction of magnetic North (H component) in nanotesla (nT) relative to an unrecorded baseline. This exact plot appeared on <a href="http://aurorawatch.lancs.ac.uk/" target="_blank">AuroraWatch UK</a>. We would normally expect cleaner data but our Lancaster site has recently been experiencing some interference.<br />
<br />
<table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto; text-align: center;"><tbody>
<tr><td style="text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgjCYkS_tUinxGkMW12UvjoQquoh2ynLE27LeFGW8WwIpvLzsa_DeQ4T-um_jCYiFbTKEcEA1JJZG8vvPqu1TpGMyGSaAGVCbr5bwIPsCU09KmT5MmxycPYBGzW08HFmMJuf8mTom3vg_Y/s1600/aurorawatch_rolling.png" imageanchor="1" style="margin-left: auto; margin-right: auto;"><img border="0" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgjCYkS_tUinxGkMW12UvjoQquoh2ynLE27LeFGW8WwIpvLzsa_DeQ4T-um_jCYiFbTKEcEA1JJZG8vvPqu1TpGMyGSaAGVCbr5bwIPsCU09KmT5MmxycPYBGzW08HFmMJuf8mTom3vg_Y/s1600/aurorawatch_rolling.png" /></a></td></tr>
<tr><td class="tr-caption" style="text-align: center;">SAMNET data from Lancaster, as used by AuroraWatch UK</td></tr>
</tbody></table>
<div class="separator" style="clear: both; text-align: center;">
</div>
<br />
<div class="separator" style="clear: both; text-align: center;">
</div>
<h3>
AuroraWatchNet test magnetometers</h3>
<div>
The plots below correspond to the same time interval as the plot above. The first plot is from my garden magnetometer.<br />
<br />
<table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto; text-align: center;"><tbody>
<tr><td style="text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjt22a5-AOZCJNxUHTnzAFiFYlGlApgouxVhUeR3gCkDhLilsakVBlUe-4SeVg55EScqWu4gnl7mb4DwfjJuIVLUPtR7t-zARrXKR4iXHQ7tJu_G8GNZBGrxjMX0Gn4-PpyHOfhHf6CyuY/s1600/awnlan1_rolling.png" imageanchor="1" style="margin-left: auto; margin-right: auto; text-align: center;"><img border="0" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjt22a5-AOZCJNxUHTnzAFiFYlGlApgouxVhUeR3gCkDhLilsakVBlUe-4SeVg55EScqWu4gnl7mb4DwfjJuIVLUPtR7t-zARrXKR4iXHQ7tJu_G8GNZBGrxjMX0Gn4-PpyHOfhHf6CyuY/s1600/awnlan1_rolling.png" /></a></td></tr>
<tr><td class="tr-caption" style="text-align: center;">AuroraWatchNet data from near Lancaster.</td></tr>
</tbody></table>
</div>
<div class="separator" style="clear: both; text-align: center;">
</div>
<div>
<br /></div>
<div>
The Y axis again records magnetic field strength but this time it is the absolute field. This magnetometer is operating in power-saving mode, turning off the sensor when not needed. This slightly increases the noise level but I hope will give one year of operation from two D cells.<br />
<br />
Another AuroraWatchNet magnetometer has been deployed almost adjacent to the SAMNET magnetometer we already use for AuroraWatch. In this system the magnetometer sensor is continually powered, resulting in reduced noise levels. The alignment with magnetic North is only approximate. </div>
<div>
<br /></div>
<div>
<br /></div>
<div>
<table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto; text-align: center;"><tbody>
<tr><td style="text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgRI440Q9mCGv_Qb5zcn5x8or2rDkQYzb51OPvtqmYkGLzsSwH7THGgkXV7VnHz5_Cwh2RWfTbi4wIaykNbRnS_ae-I61bo1yoFuKaJmF6JIR00YNiRBlkm2JjSUMs6PrTPsFavUaQyv3A/s1600/awnlan3_rolling.png" imageanchor="1" style="margin-left: auto; margin-right: auto; text-align: center;"><img border="0" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgRI440Q9mCGv_Qb5zcn5x8or2rDkQYzb51OPvtqmYkGLzsSwH7THGgkXV7VnHz5_Cwh2RWfTbi4wIaykNbRnS_ae-I61bo1yoFuKaJmF6JIR00YNiRBlkm2JjSUMs6PrTPsFavUaQyv3A/s1600/awnlan3_rolling.png" /></a></td></tr>
<tr><td class="tr-caption" style="text-align: center;">AuroraWatchNet data from the Lancaster University field station.</td></tr>
</tbody></table>
</div>
<div>
<br /></div>
<h3>
Conclusion</h3>
The AuroraWatchNet magnetometers are sufficiently sensitive and stable to make useful contributions for auroral detection and citizen science. My experience with the garden magnetometer suggests that in many cases performance will be limited by the noise at the site, rather than the magnetometer.</div>
</div>
Steve Marplehttp://www.blogger.com/profile/15582257560382364498noreply@blogger.com0tag:blogger.com,1999:blog-252642453295590453.post-8656114466991855632013-01-10T22:40:00.000+00:002013-02-08T14:38:20.627+00:00UART-USB converter for mixed 3.3V/5V operation<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjbGPBAExvonKMZaYDQ1kEqq-v8ZNhXf1Ae2b5aOw95w4ppbXXsai-nFbX7L_QU2RuYmemYaNT8ODt6uc6aRp0OQQlrZ4YtbUqZuqdRM6XdPqC7CdWikj578F63cnBzjogPirivWcn0tNk/s1600/MCP2200_example.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="222" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjbGPBAExvonKMZaYDQ1kEqq-v8ZNhXf1Ae2b5aOw95w4ppbXXsai-nFbX7L_QU2RuYmemYaNT8ODt6uc6aRp0OQQlrZ4YtbUqZuqdRM6XdPqC7CdWikj578F63cnBzjogPirivWcn0tNk/s400/MCP2200_example.png" width="400" /></a></div>
<div class="separator" style="clear: both; text-align: center;">
</div>
<br />
For some future projects I have in mind I would like to add a USB interface to my <a href="http://blog.stevemarple.co.uk/search/label/Calunium" target="_blank">Calunium</a> development board, rather than using the <a href="https://www.sparkfun.com/products/9873" target="_blank">Sparkfun FTDI basic breakout</a> board that I have been using. One of the key features is that it should work seamlessly with the jumper-selectable 3.3V or 5V operation offered by <a href="http://blog.stevemarple.co.uk/2012/12/calunium-version-2.html" target="_blank">Calunium v2</a>, which is most easily accomplished by a self-powered design; <a href="http://arduino.cc/blog/2007/09/07/enabling-auto-reset-on-the-arduino-ng-and-serial-boards/" target="_blank">auto-reset</a> is a bonus.<br />
<br />
Microchip offer the MCP2200 USB interface in a choice of packages including a 20 pin SOIC package which is very convenient for home assembly of PCBs. I found conflicting information about the MCP2200 so I prototyped the above circuit to check that mixed 3.3V/5V operation was ok. VUSB (pin 17) is permanently connected to 3.3V so that the USB signalling levels are always correct. The VDD supply pin is connected to IOREF on my Calunium board, which is either 3.3V or 5V, depending on the position of the voltage selection jumper, ensuring that the logic levels for the MCP2200 match those used by the microcontroller. I also wanted to test whether the MCP2200 would auto-reset the microcontroller. The MCP2200 doesn't have a DTR output pin which is how modern Arduinos reset the microcontroller for uploading a new sketch. Fortunately the MCP2200 does have an RTS output, and as that was the method used by older Arduinos I connected that to /RESET via a 100nF capacitor.<br />
<br />
After building the circuit and connecting it to my Ubuntu 10.04 Linux computer the MCP2200 was immediately recognised, I used the dmesg command to see this. Ubuntu automatically created the /dev/ttyACM0 device file, no configuration changes were needed at all. Using gtkterm I was able to toggle the state of the RTS pin. I then hooked up the circuit to one of my Calunium microcontroller boards and was able to upload sketches without needing to press the reset button, in both 3.3V and 5V operation modes. I haven't tested this from Windows but from what I've read on the Microchip forum I suspect it won't work due to deficiencies in the Windows driver.<br />
<br />
The datasheet suggests that a ceramic resonator can be used instead of the crystal and load capacitors. I found it did work even though the resonator's frequency tolerance was 0.5%, not 0.25% as required by the USB specification. As I can't source any resonators which meet the specification I'll be using a 12MHz crystal.<br />
<br />
The MCP2200 wasn't my first choice; the obvious part was the FTDI FT232RL, although I had reservations about working with an SSOP package. Further investigation revealed that 3.3V operation requires an external crystal, but to use one isn't straightforward. On first use the FT232RL must be powered from >= 4.0V and a custom utility (not openly supplied?) is needed to select the external crystal. The standard MPROG utility cannot be used for this. I am aware that the FT232RL could be powered from the USB 5V supply but I am not a big fan of powering devices simultaneously from both internal and external supplies. One of the future projects is to replace hardware which is powered from multiple sources and is very susceptible to damage from being powered up/down incorrectly.<br />
<br />
In conclusion, this is a very useful USB converter for Linux users and for final designs. Windows users who wish to regularly upload new firmware via a bootloader may find it less useful.<br />
<br />
<br />Steve Marplehttp://www.blogger.com/profile/15582257560382364498noreply@blogger.com1