New AAVSO Data Submission Formats & Guidelines
You have heard the rumors and wondered when the day would finally
arrive. Well it's here and we are happy to announce new formats for submitting data to the AAVSO. The format was originally created over 13 years ago and,
for the most part, it has changed very little since. However, times have
changed and the format needed an update to account for new observing technologies, the desire for more information from our professional colleagues and
the expansion of the observing program.
- The new format specifications describe how you should upload a file of multiple observations. If you enter observations individually, we provide the forms for you.
- PCObs (for visual observers) and many photometry software packages (for CCD observers) will export the data according to the new specifications automatically
- The entire GCVS catalog has been added to the stars in our program (except the NSV stars, which we'll get loaded soon)
- The harvard designation and lots of comment codes have been retired from service
- After July 1, 2008, if you try to submit data in a *new* format that is not one of the two AAVSO official formats, it will not be supported.
Behold, the new formats!
There are two new formats for submitting multiple observations to the AAVSO at one time.
- Visual: This format is used for visual and photographic observations
- Extended: This format is used for photometric observations (CCD, PEP, etc.)
The formats are very similar. However, the extended format includes more fields for the additional information we would like to record about photometric observations. Both formats are ASCII text based and delimited based on a
character of your
choice (as opposed to the original format which was column deliminated).
The vast majority of you will not have to worry about the details of the new format. If you use the new PCObs version 3.0 or individual submission via WebObs to submit observations, then our software does all the work for you. The new interfaces for those programs look very similar to
the previous versions and they work in the same manner. However, if you prefer to draft your own submission reports using custom software or programs like Excel, then this new version will be much easier for you to program. The fields are now
exclusive
and don't overlap. For visual reports, the number of required fields per record has also decreased.
Now let's go through the major changes between the previous and new formats:
Exit Harvard Designation, Enter AUID (AAVSO Unique Identifier)
The Harvard Designation has served us well. Being tied to a position in the sky has its advantages. However, it also has its problems - especially with high proper motion stars
(such as Barnard's Star, V2500 Oph) that are now not anywhere close to where their epoch 1900 coordinates suggest. Also, the structure of the designation limits how many designations we can assign. Basically, we can only have 26 stars within a common RA/Declination
(1234+56A - 1234+56Z). Now that tens of thousands of variable stars are known, and hundreds of thousands are expected to be discovered as modern surveys come online, we need a more flexible system.
The AAVSO Unique Identifier (AUID) is in an alphanumeric "license plate": 000-XXX-000 where the 0's are 0-9 and the X's A-Z. This gives us 17,576,000,000 possible combinations - enough to
keep us busy for a long time! Every star in
the AAVSO International Database has had an AUID assigned. As we add new stars, we will assign new AUIDs.
You will never be forced to use the AUID for anything! We have also begun a project to make sure that all "names" of stars in the database are also unique. That way you can always use the name of a star to submit data, plot a light curve, make a chart, etc. The AUID is harder to memorize for most GCVS stars (but easier than 1RXJ053234.9+624755!).
99% of the stars in our validation file have unique names right now. The ones without them are usually comparison stars (ex: "comp 123 AB") and we are in the process of assigning them proper, unique names. These new names will not be limited
to 12 characters, either.
Beginning January 15, 2008, Harvard Designations are no longer being assigned to stars in our program. Instead, we now assign AUIDs and unique names.
The Validation File List
The Validation File has also seen a makeover in preparation for this new format. It is now known as the The AAVSO Validation List. And as we mentioned, the Harvard
Designation is being replaced by the
AUID. In addition, we are greatly expanding the stars
in our program with the policy to accept observations of any variable star in the sky. The first part of this project involved importing around 33,000 stars in the General
Catalogue of Variable Stars which were not already in our program. You can submit observations on these stars immediately.
A procedure has been created to add new variable stars to our validation file in a quick fashion. When you submit an observation on a star not in our program through WebObs, you will be prompted to either wait 3 business days for us to add
it to the Validation List (if the star exists in VSX) or you will be prompted to send information to aavso@aavso.org so we can add it to our list and decide whether it should be added to VSX (only true variables should be added to VSX, not
suspects). If you are not using WebObs, e-mail aavso@aavso.org with info on the star you want to add and we will get back to you within 3
business days of your request! Our eventual goal is to completely automate this process, but we are probably a year or so away from seeing that dream come to fruition.
Our validation file/list has been greatly expanded and now can be updated in a much quicker fashion.
New Collection of Comment Codes
The previous list of comment codes included descriptions that could overlap (such as K, J and T). Also, some codes simply were not needed anymore (ex: since the introduction of the comparison star field we don't need the J and T comment
code
information anymore). Below is the list of new and retained comment codes. Notice that the description of some of the retained codes have been tweaked.
- B: Sky is bright, moon, twilight, light pollution, aurorae
- U: Clouds, dust, smoke, haze, etc.
- W: Poor seeing
- L: Low in the sky, near horizon, in trees, obstructed view
- D: Unusual Activity (fading, flare, bizarre behavior, etc.) (new)
- Y: Outburst
- K: Non-AAVSO chart
- S: Comparison sequence problem
- Z: Magnitude of star uncertain
- I: Identification of star uncertain
- V: Faint star, near observing limit, only glimpsed
Like a popular athlete, we have chosen to retire the following codes. They will be retained in the database and these jersey numbers codes will never be used again. But new observations
should not use any of these comment codes:
- A: AAVSO Atlas
- F: Unconventional method (out of focus, visual photometer, etc.)
- G: Non-AAVSO chart with Guide Star Catalog magnitudes
- H: Haze, mist, fog
- J: Non-AAVSO chart with HIPPARCOS magnitudes
- M: Moon present and inteferes
- N: Angle, position angle
- O: "Other" comment
- R: Color comment (star is red, blue, etc.)
- T: Non-AAVSO chart with Tycho magnitudes
Uncertain about Uncertain?
Previously, in the magnitude field, one would put a colon (:) after the magnitude to denote uncertainty about the magnitude estimate. This was removed in favor of using the Z comment code.
Comment codes have changed and been updated. Also, a significant number have been retired and are no longer accepted.
Comparison Stars
There are now separate fields for the two comparison stars used to bracket the magnitude estimate. Please only include one comparison star in each field. If you do use a third star, place it in the comments field along with an explanation
as to why you needed a third star.
Submitting Observations
Reports formatted in the Visual format, may be uploaded as a file to the Visual batch upload page via WebObs in the Blue&Gold section of the site. They may also be submitted by e-mailing them to observations@aavso.org. Extended reports may only be submitted via the CCD batch upload tool in
BlueGold's WebObs function.
Software That Supports the New Formats
PCObs 3.0 (programmed by volunteer Len Abbey) is a Windows-based data entry tool that will export a file in the Visual format.
The following photometric software packages have agreed to export data to a file in the Extended format. (Note that you will need to upgrade to the latest version.) This means that in the future you can load files from these programs
directly into BlueGold/Webobs without any additional formatting on your part. Here is an updated list of software support for the new AAVSO format.
 |
| Screen shot of AIP4Win Version 2.2.0 which will be available as an update soon. Click image for full view. |
Many software vendors have agreed to support the new Extended format in upcoming releases. This will make submitting time series and large batches of observations much
easier.
What About the Old Format?
The old format (now referred to as our legacy format) will still be supported until July 1, 2008. Observations in this format may be submitted by sending them
in an
e-mail to observations@aavso.org or by using the upload tool in BlueGold's WebObs function. We strongly encourage you to move to the new format! By using the old format you will not be taking advantage of many of the new fields,
comment codes and other new features in the new formats. This makes your data less valuable in the database compared to the same data in the new formats.