Exploring Swarm data
with VirES

Darren Chaddock  Lead Developer - University of Calgary
January 11th, 2023


VirES for Swarm is a highly interactive data manipulation and retrieval interface for the ESA Swarm constellation mission products. It includes tools for studying various geomagnetic models by comparing them to the Swarm satellite measurements at given space weather and ionospheric conditions.

In Swarm-Aurora's latest update, the Conjunction Finder interface now has the ability to quickly open up VirES and explore Swarm data in more detail. Below the map, you'll see a "Open in VirES" button. Several preset loading states have been integrated with Swarm-Aurora to help you look at certain types of Swarm data in a single click. You'll see those in the dropdown menu for the "Open in VirES" button.

If you'd like to load VirES with a different state more to your liking, you can use a custom file. To generate one of these yourself, use the "Workspace -> Export" navigation item in the VirES web interface. Note that when using a custom file, only the timeframe will be set by Swarm-Aurora before opening VirES. No adjustments to the Swarm A/B/C data product combinations are made.

Other minor updates:

  • Keogram browser - added a UT time bar to the top of the keogram area
  • Keogram browser - added data loading UI component (same as Conjunction Browser)

As always, if you have any suggestions, feature requests, usability improvements, or additional summary data that you'd like included, feel free to contact us.

- Darren Chaddock


Integration with AuroraX,
and faster loading times

Darren Chaddock  Lead Developer - University of Calgary
August 20th, 2021


AuroraX is a project working towards being the world's first and foremost data platform for auroral science. One of the first tools we've developed for that project is called the Conjunction Search. This is a progression from Swarm-Aurora's Conjunction Finder, where-by the goal is to identify conjunctions automatically using a search engine. Now that the AuroraX Conjunction Search tool is stable, we have added the ability to load search results directly in Swarm-Aurora for quick evaluation. After a conjunction search is performed, click the "Open All Results" button in the Swarm-Aurora toolbar above the "Search" button. Try it here.

In pursuit of providing web tools that are fast to use, we have deployed Swarm-Aurora on the cloud in four locations: Calgary, US west coast, US east coast, and Europe. Further, we have made some improvements and reduced summary data loading times significantly. You can choose to turn off keogram preloading using a new toggle below the Conjunction Finder map. This will make loading data slightly faster initially, but not when moving around different times in a day.

Lastly, we have added SAMPEX and all DMSP satellites to the Conjunction Finder.

If you have any suggestions, feature requests, usability improvements, or additional summary data that you'd like included, feel free to contact us.

- Darren Chaddock


TREx and Zero-day Lag

Darren Chaddock  Lead Developer - University of Calgary
March 20th, 2020


Today we are releasing a new version of the Swarm-Aurora website which most notably includes the addition of several TREx datasets and the ability to browse data from the previous night! Further, we have back-filled existing datasets to the start of their operational times (ie. THEMIS ASI now goes back to 2006).

Over the past several years, the Auroral Imaging Group at the University of Calgary has been working on several new ground-based auroral instrument arrays as part of the Transition Region Explorer (TREx) project. TREx includes arrays of GNSS systems, imaging riometers, spectrographs, and several different kinds of all-sky cameras (427.8nm Blueline, 844.6nm near-infrared, and broadband RGB colour). Further information about the project can be found here.

As our group moves into the final months of the TREx project, we are currently deploying the remaining instruments and making high-resolution and summary data available to the science community. For this release of Swarm-Aurora, we have incorporated the TREx Blueline, Near-Infrared, RGB Full-Colour, and Spectrograph instrument arrays into the Swarm-Aurora website. All high-resolution data is freely available on our group's Data Landing Page, obtainable using HTTP/HTTPS, anonymous FTP, or Rsync.

Additionally, we have added a significant amount of summary data for the existing datasets in Swarm-Aurora. Satellite ephemeris has been back-filled to each satellite's start date in SSCWeb, and THEMIS ground-based all-sky imager data is now available going back to 2006. AGO and MIRACLE include summary data form 2017 and 2018 and we will be working on 2019 data over the next several months. Swarm EFI keograms and field-aligned current data from 2017-present now exists as well. You can use the Data Availability page to see what summary data we have. By default, it only shows the Satellite Ephemeris, so you can use the dropdowns to load the availability for other datasets of interest.

Lastly, the Swarm-Aurora platform is now displaying data in a pseudo real-time manner. Our population scripts update the website and data tree every 2 hours with any new data that has been collected during that time. This means that you can browse data using the Conjunction Finder and Keogram Browser in the morning and look at last night's data!

As usual, if you have any suggestions, feature requests, usability improvements, or additional summary data that you'd like included, feel free to contact us.

- Darren Chaddock


Data Availability, and more

Maryam Sohrabi  Assistant Developer - University of Calgary
December 6th, 2019


Over the past few months we have released a host of new improvements and additions to Swarm-Aurora. User interface improvements aimed at creating a more intuitive user experience, include switching to drop-down project/satellite selection menus in the Conjunction Finder and Keogram Browser.

In addition to the user interface updates, we now have several new features:

  • Data Availability Viewer - This new visual tool will help users quickly determine what data is available for each project in a monthly or yearly timeframe. This page has replaced the data availability pop-ups that were available throughout the previous version of Swarm-Aurora. The data tree remains open for regular directory browsing.
  • Polar MLT Keogram Browser - 24-hour UT plots and hourly plots for fixed MLT, from the Polar satellite 1998 data, can be found in the new MLT Keogram Browser. This browser is similar to the existing fixed station Keogram Browser that Swarm-Aurora users will be familiar with.
  • FAST ESA IES keograms - Hourly plot keogram for 1998 data are available in the Conjunction Finder. To view them, select a date in 1998 and ensure FAST is checked on in the HEO dropdown menu.

We hope that these improvements and additions prove useful to the scientific community. If you have any suggestions, feature requests, usability improvements, or additional summary data that you'd like included, feel free to contact us.

- Maryam Sohrabi


Satellite Finder

Darren Chaddock  Lead Developer - University of Calgary
June 25th, 2017


Today we are releasing the latest version of Swarm-Aurora with a couple of major additions and improvements. Most noticeably is the addition of the "Satellite Finder" tool. You can find this as a new item in the top navigation bar. The Satellite Finder allows you to easily and quickly find out where a particular satellite is in real-time. This way, if you're wondering if Swarm or ePOP is going to fly over you in the next 20 minutes, just enable whichever satellite you're interested in and watch the map constantly update every minute. Also, we've included a quick-link button that drops a pin on Google Maps of the current satellite you're tracking. This way you can use a more in-depth map that what we have currently.

When we were building this new tool, we were thinking primarily of mobile users, and specifically citizen science communities (ie. aurora chasers like the Alberta Aurora Chasers). It could be used to help citizen scientists schedule their outdoor aurora chasing trips better and increase the chance of capturing interesting events with Swarm overpasses.

Aside from the Satellite Finder, we have uploaded 2013 data for the THEMIS ground-based ASI array in the Conjunction Finder (along with all available satellite footprint data). We've extended our tutorials to include ones for the Keogram Browser Satellite Finder. Also, we've included dynamically updating "Data Availability" tables for you to see exactly what timeframes of data are available to view. You can see these tables by clicking on the button below the maps on the Conjunction and Satellite Finder, and, on the left side of the Keogram Browser.

To better visualize the instrument operations of ePOP, we are now dynamically changing the colour of the ePOP satellite trace in the Conjunction Finder based on if instruments are running. Now you don't have to think quite as hard when looking for ePOP data - just look for a green trace and you know there is some data and you can look at the ePOP Instrument Operations table below the map for further information.

Lastly, in the Conjunction Finder we've included the ability for you to export/import events that you have shared or received from colleagues. If you find an interesting event and want to save it or show it to someone else, you can use this import/export functionality. You can find the buttons for this feature below the map (also outlined in the tutorial). Also, you can use the "Custom Import" feature to import a list of dates that you have from another study. For example, you have a list of dates that there is patchy-pulsating aurora in a THEMIS ASI and you want to see if there are any satellite conjunctions. Simply upload your list of dates (see icon beside the "Custom Import" button for information about the formatting of this file) and it'll populate a drop-down list for you to cycle through.

If you have any suggestions, feature requests, usability improvements, or additional summary data that you'd like included, feel free to contact us.

- Darren Chaddock


2016 data and
Swarm-Aurora On The Go

Darren Chaddock  Lead Developer - University of Calgary
February 8th, 2017


Today, we are launching the latest version of Swarm-Aurora. Over the past month, we have made a couple of large additions including the integration of 2016 data! This was a common request from the science community while discussing and presenting at the AGU Fall Meeting in December and the 2nd Workshop. Keeping up with our focus of only showing summary data for when the instruments were operating and raw data is available, most of the ASIs will only have 2016 up to about October or November of 2016. This is mainly applicable to THEMIS-ASI, RGB, and REGO, since we haven't received the raw data quite yet. That should start to be available in approximately March. Also, we are currently working hard to get 2016 data for MIRACLE integrated by the next release (March 1).

To compliment the web-based Conjunction Finder and Keogram Browser tools (and to provide quicker data browsing), we have developed a desktop application which allows you to browse summary data while 100% offline. We are calling this "Swarm-Aurora On The Go" and you can find more information here. This application is cross-platform and we provide you with the ability to download custom configurations of summary data, or whole months/years of data using pre-packaged zips.

Next up on our list of tasks is most notably the integration of a ~2 day lag in the summary data. Currently, we are only providing summary data for 2014 through the end of 2016, but, we'd like users to be able to browse the most recent summary data. Our next release is planned to be March 1st, so check back in then to see more new features.

If you have any suggestions, feature requests, usability improvements, or additional summary data that you'd like included, feel free to contact us.

- Darren Chaddock


Welcome to
Swarm-Aurora

Eric Donovan  University of Calgary
February 7th, 2017


The launch of the Swarm constellation occurred during an exciting time for auroral science. Near-global networks of ground-based instruments are providing an unprecedented view of the multi-scale ionospheric consequences of geospace dynamics and magnetospheric-ionospheric coupling. This extensive network provides real time observations of the night sky aurora, providing a window into the dynamics of the near-space environment. For more than forty years, the University of Calgary has been a world leader in auroral observations. Today, with funding from the Canadian Space Agency, we operate the world's largest network of ground-based auroral imagers. This network consists of imagers of different types, including our full-color Rainbow All-Sky Imagers (ASIs). Rainbow ASIs produce millions of images every year.

The Swarm-Aurora project was designed to facilitated and drive the use of Swarm in auroral science and push Swarm beyond its primary mission objective to become a key instrument in auroral science research. The primary objective of Swarm-Aurora is to build a bridge between Swarm data, the Swarm science community, and optical images of the aurora collected primarily by ground-based All-Sky Imagers (ASIs).

Researchers must be able to easily identify Swarm overflights of ASIs that were capturing images of aurora of interest. As well, for each such overflight, it is in general necessary to relate the electric and magnetic fields observed by the satellite to the aurora structures and dynamics as captured by the ASI. The Swarm-Aurora project was developed to eliminate the uncertainty and drastically reduce the time needed to do a preliminary survey of Swarm and ground-based instruments for investigating auroral phenomena. In one environment, which can be downloaded onto a laptop or desktop computer therefore eliminating the need for an internet connection, it is possible to have years of data from many different observing missions in one place. Researchers will be able to quickly scan through years of data within minutes, quickly identifying events that meet their specific research goals.

Swarm-Aurora will carry out the groundwork necessary to enhance the value of Swarm an auroral research mission. Over 16 months, Swarm Aurora will:

  1. Bring a large fraction of the world's auroral imaging capacity to Swarm as efficiently as possible
  2. Enable researchers to identify scientifically valuable Swarm overflights of ASIs
  3. Enable users to utilize ASI data together with Swarm data
  4. Demonstrate the efficacy of the developed tools for addressing cutting edge science
  5. Engage auroral researchers in developing a roadmap for ongoing Swarm Aurora activity

Looking forward, we hope to include ephemeris from more satellites such as GOES, LANL, and others, and some cursory summary data from some or all of those. As well, we hope to include auroral/airglow networks beyond the "first five". These should include imagers operated by SRI and Boston University, the Alaska Geophysics Institute, the Polar Research Institute of China, UNIS and the University of Oslo, IRFU, and the University of Nagoya. Furthermore, we hope to include photometer summary data and possibly riometer summary data.

A key objective is to have an effective research enabling tool. This means we are committed to providing a user experience that is as fast as possible. This means we will be keeping the summary data very light. Keep in mind, though, that we are in the age of Big Data, and we expect in the end that a day of summary data will be hundreds of MBytes.

To develop this browser, we have drawn on a lot of previous work, including the development of GAIA (gaia-vxo.org) by Emma Spanswick, Brian Jackel, Mikko Syrjaesuo, and Steve Marple, and the Cluster Ground-Based Working Group work carried out by Paul Eglitis, Kirsti Kauristie, Hermann Opgenoorth, and Mike Lockwood. We hope to develop an API to allow interleaving this effort with SuperMag, SuperDARN, and GAIA.

- Eric Donovan