How to recreate plotted weather charts with observations from the Daily Weather Report – yes it’s rather a mouthful as titles go but it’s what I did during Christmas 2017. The Daily Weather Reports [DWR] in question are the ones that the Met Office have very kindly scanned and made available on their website. The daily reports extend back to September of 1860, but for a start I’m only interested in the years from around 1960 to 1972, after that I have my own 6 hourly observational data that I bought from Weather Graphics some years ago and have kept updated by means of OGIMET in recent years. So if you want interested at looking at a plotted synoptic chart for a particular day from January 1963, you could approach the Met Office and ask them to provide you with all the SYNOP observations for the hour that you’re interested in, I haven’t done that personally, because I think the cost would be prohibitive. But there is another way, and that’s to download the PDF of the DWR for January 1963 and extract the 55 observations that are contained within it and plot your own chart, and that’s exactly what I did:
- Download the PDF of the DWR for the month of January 1963 from the Met Office (~67 MB).
- Cut and paste the observations for the hour that you are interested in (00, 06, 12 or 18) as a JPEG from the PDF.
- Use a good OCR application to create a text file from the JPEG.
- Write an application to allow you to edit the text file returned by the OCR application to verify and edit each observation.
- Use the application to convert the old format SYNOP into the new format.
- Download the reanalysis MSLP pressure data for 1963 (~19 MB) from the NCEP site, convert the NetCDF format into plain text using NCDUMP, parse that gridded text into individual MSLP data files for each main synoptic hour for that year.
- Inject the MSLP 2.5° x 2.5° gridded data values for 18 UTC on the 2nd of January 1963 from the data file you created in step 6 as a background field to improve the MSLP contouring, along with all the SYNOP observations that you created in step 5 into your SYNOP viewer, ensuring that you have all the locations of the old observing sites (such as Spurn Head and Cape Wrath) in your stations database so you can plot them!
- Generate a screenshot of the British Isles for 18 UTC and add it into you blog.
And that readers is all there is to it.
Optical Character Recognition
I have found one thing from this exercise and that OCR software is not a great deal better than it was 20 years ago when I first used it. None of the free and online OCR web services work at all well, and none of them allowed you to choose numeric only input. Some of the results were so bad that the numeric results have been converted into the equivalent of a piece of Shakespeare in the Infinite monkey theorem! I tried to sharpen and reduce the number of colours to no avail, I cut and pasted and saved as high quality JPEG, PNG, TIFF and PDF without any improvement in character recognition. Just at the point of giving it all up as a bad job I decided to download a trial of Abbyy PDF Creator+ and at first found the results were equally as poor, until I noticed that if I set the language to digits the results were very much better, probably around 90% of the numeric characters in the observations were now correct.
The final result
And so here is the finished product (fig 2), exactly 54 years to the day since the observations were made. I was still a little young to be involved with making one of the 55 observations quite yet, but I did spot a typo in the original DWR, it occurred in the 03715 Rhoose observation, and although I would for a bit of fun like to submit a correction to it to the Met Office help desk, they’d probably not see the funny side of it 54 years after the event as I do!
Of course the perfect answer to my conundrum would be for the Met Office to make all their archived SYNOP observations freely accessible, but I don’t see that happening anytime soon, but if you’re as daft as I am, and follow the steps that I’ve outlined above, you too could step back in time and recreate a plotted weather chart from the 1960’s.