The Videotex Database - submit your pages now!

When I started viewdata.org.uk (and teletext.org.uk), I just uploaded the pages and databases I had as-is, and had my scripts deal with them on an as-accessed basis.  This is because I wanted to preserve the data as much as possible - any translation to a new format (such as JPEGs) would inevitably lose data, as well as context.

As time has moved on, and as the variety of data formats I have had to deal with has proliferated, this has increasingly become somewhat unwieldy. I decided, therefore, to try and rationalise things somewhat.

Each of the various file formats I was dealing with had different properties. Each had strengths, and each had weaknesses.  I could not decide on a single common format to try and convert files into.

Rather than create a new "perfect" file format, I decided therefore to store the frames within a database.  By having a primary table for the page content and certain static data, and a separate table for meta-data, any particular properties a particular file format had could be accommodated.

Once the data is held within a standardised database, of course, it makes it much easier to access it and use it from many different applications.  The first, and most obvious, is the ability to search across the entire database for key words or phrases. This is implemented on the front page of the database.

The main in-browser viewer for the saved pages implements a timeline function, where you can see how a given page has changed over time.  See, for example, the CEEFAX news headlines.

And of course, for viewdata pages, once can implement a dial-up host, so 1980s terminals can connect directly into the service and browse it exactly as they did at the time.  (This is mostly done, just pending further tidying up!)

Currently the database contains page data I have collected myself or already been sent. However I am aware that there is a vast amount more out there.  Jason Robertson has been amazing at rescuing teletext pages off old video tapes, and I know of at least one previous Prestel IP that has a massive archive of pages still extant, albeit sat on very old hardware.  I've got part of The Gnome At Home, and I know the rest still exists.

This week's task (one of the various "I'll do something" for Retrochallenge 2018/09) was to create a page for viewers to directly submit their pages to the database.  This is now complete!  It actually places the data into a queue, after briefly validating it, so it can be checked and added later.  I would welcome any contributions, anything from a single frame to a complete service backup!  If you need help, feel free to drop me a line.