Archiving sites

When we rebuilt the openly local map we knew that there were hundreds of hyperlocal sites, Facebook pages and Twitter accounts out there that weren’t on the map.  And that the map database contained hundreds of sites that had ceased publishing as natural wastage since 2009 in a fast moving sector where people are testing a myriad of business models.  Based on our recent work we judge that there are between 1,500 and 2,500 active hyperlocal sites in the UK of which we have 511 mapped.

Through some desk based exercises in the last couple of months, Finding Scottish Hyperlocals, Devon & Somerset – putting about 100 hyperlocal sites on the map and the Wakefield experiment we have added 209 sites to the map – not ‘new’ sites, just sites that had been around for ages but hadn’t got on the map. Our exercise covered barely 10% of the population. So based on this exercise we are cautiously confident that there are at least 1,000 other sites out there waiting to be added.

We have known for a long time that the OL map had lots of dead wood in it – sites that started and then shut up shop for some reason as part of natural wastage since 2009 – for instance the Daily Mail’s ‘Local People’ sites that ‘restructured’ in 2013 removing 122 sites from the database. So I’ve spent the past week reviewing all the sites on local Web List and archiving sites.

When we took the map over from Openly Local we imported 716 sites.  We added in around 209 in our desk based exercises.  This led to a grand total of 925 sites.

Every one of the 925 sites has been checked to see if it still exists and is occasionally updated, i.e is active somehow.  From this process the current state of the database is:

  • 925 sites at start of exercise
  • 372 sites no longer active (Archived but findable through search)
  • 42 sites that seem to have become spam or malware factories as their domains have been taken over – we have unpublished these to protect the unwary
  • 511 active hyperlocals displayed on map

This leaves 511 active hyperlocal sites on the map in the UK and ROI (that we know of).  These changes are natural wastage over five years that we have been overdue in catching up with.
Will and I have made a decision to not delete any sites in the list that are no longer active, but to move them into Archive status. This means that while they don’t appear on the map or lists of sites, they are still searchable. In the future if they do become active again we can just change the status of the site back to live.

The criteria I used when reviewing the sites was very roughly this:


Dead sites – Does the site exist yes or no, if the site returns a 404 or server doesn’t respond then I archive it.

Changed sites – If the site exists then I check is it the original site, if it isn’t then I’ll unpublish it.

Spam/Link Bait I have found that some domains have expired and been hoovered up, these sites are now something completely different, some are legitimate sites for new things but a vast majority of them are spam / link bait sites. Will and I will need to discuss what is the best way forward with these, I’d hate to just delete them so maybe we just need to remove the URL so people can’t click through. If you have a view on what we should do, then let us know.

Active sites – if it is the original site, I check to see if it has been updated in roughly the last 3 months, I take a varying view on this depending on the type of site:

News-y sites  So if it is a site that looks news led i.e. latest stories on the front page, or one that I know used to publish every day or week I look to see when the last post was published. If it has been updated recently I leave it, if it hasn’t then I move it to archive status.

Community information – If the site is more a community information site than a news site and hasn’t been updated then I leave it.

One thing that appeared during this exercise is that custom sites with domains seemed to stop publishing and die off more than sites that used the available free platforms like  Blogger & WordPress. Maybe the hassle of having to keep the site updated technically as well as with content makes it less fun? More analysis would be needed to draw any real conclusion from this.

All of this isn’t fool proof, so if I have got your site wrong or you think we should be doing it differently then get in touch with us.

Devon and Somerset – putting about 100 hyperlocal sites on the map

The old OpenlyLocal map only had about 5 sites for Devon, mainly the defunct ‘Local People’ franchise. This couldn’t be right for England’s second largest county. Devon has a strong tradition of local pride in place and its rural nature meant there are obvious advantages to having websites for local civic life. As well as a healthy tourist industry for which search now seems to be mainly online.

We’ve been writing about our work to improve the LocalWebList map.  So to fill in this blank space on the map I started Googling place names and following links around Dartmoor, where I knew of one independent site Talk About Local worked with some years ago in Moretonhampstead. After a while, this led me to a resource on the Devon County council website attempting to list Devon’s local websites.   Which in an exercise like this is hitting paydirt.  So I followed the links on the DCC site, make a quick assessment of what I find there, add it to LocalWebList, then in each site look for outgoing links to other hyperlocal sites.  I haven’t yet finished but have added in over 100 sites so far, including some in North West Somerset.  Some observations on the sites I mapped in this exercise:

they are nearly all village sites, orientated around the parish council or other local committee such as a hall, many dating from five years ago or more, with occasional updating (monthly or quarterly it feels to me, some annually). This can be very rich (see this site for Belstone) but this type tends not to have a journalistic ‘news’ component.

almost all these sites serve a basic transparency role that could only otherwise be served by visiting in person the parish clerk at their home or the church hall. They provide on demand information resource that otherwise is constrained by time, place and networks. Given the highly rural nature of these areas with low population densities having an online information point adds considerable value if it prevents a journey or otherwise lowers the threshold to getting hold of information.

they also provide basic information about village life that to my mind contributes to bridging social capital by connecting people together (church, institutes, clubs for parents, activities for older people, nature). And would help visitors and new arrivals as a community welcome pack.

some are quite eccentric in web design terms, there are one or two mini groups done by the same web developer for adjacent villages/parishes.

these sites are basic village information sites, run by local people. i have included one or two (very) small town websites that are clearly locally important and more professional, but aren’t

there is often a mild to moderate tourism element, given the areas involved. I have not been including out-and-out tourism sites with no civic nor journalistic components.

maybe only one or two of these sites have interesting discussion or place to challenge civic authority on them.

active ‘news’ on these sites as a newsy person would recognise it is rare.  There is a scattering across Devon of relatively weak websites from small local Tindle group papers.

I would expect there to be some Facebook action that runs in parallel for some of these sites, but it isn’t clearly flagged up from the sites themselves.  There is a very good example in UpLyme.  I have come across only a few references to Twitter from these sites such as the very good Dartmoor NPA   and Belstone again but there must be more hyperlocal Twitter action out there.

none of the sites I have mapped in this exercise seem to be as active as the UK’s class leading village website up in the Peak District.

the tedium of this exercise (as Johnson said of making dictionaries, it is dull work) was lessened by the discovery of my favourite place name Zeal Monachorum.

There’s still more to do, I haven’t yet finished going through the Devon CC list.  And I notice that in Cornwall there is a list of parish clerk contact details which often gives away the presence of a local website with a bespoke URL in the clerk’s email address.  So I shall plough through this when I have time.  Many counties publish parish clerk/council details so this could be a route to follow elsewhere.

Finding Scottish hyperlocals

Living and working in rural Aberdeenshire, where I run my own hyperlocal site, I am the official Scottish correspondent for Talk About Local. Following on from Will’s post about filling in the white space around Wakefield on the Local Web List map I thought I’d share my experiences adding in sites from Scotland.

Way back in the history of Talk About Local, Will found a great site that listed lots of community websites across Scotland, the fantastically named Sadly now the site seems to be more of a business directory and the list of local sites is nowhere to be seen.

Fortunately the wayback machine has a copy of the local sites page

I am using this list as a starting point to find local sites to add to the map. The sites listed on here will, if they are still live, have a decent history and archive of content. I’ve added around 50 sites using this method, when we took the data from Openly Local there were less than 20 sites listed for Scotland, I now have 64.

The task isn’t an easy one and can be mind numbingly boring at times, working through the list you have to click the link which takes you to the archive page, then you need to go to the current site, if it still exists, a great number don’t.

You come across sites which look like they have great potential but when you go to the live URL, it is ‘reserved for future use’, then you find a gem like

Once you have found the live site you need to review it, I use some basic criteria

  • Is the site still in the spirit of a hyperlocal site? There are a lot of early hyperlocal sites that have changed to glossy corporate sites or affiliate link directories sites. I discount these sites, unless I can find any hyperlocal information.
  • Is the site dead? If it is dead but not been hoovered up by someone and still has the original content on there then I add it for posterity.

there are no real short cuts for us to add sites to the list, we have to fill the form in for each site we find, this ensures that the data is accurate and valuable to the public and our academic colleagues. We do have to make some assumptions about the radius the site covers and the description of the area covered.

Sometimes when you are looking at these sites, you get drawn in to them, there is something on the site that is just too interesting to not read, then you end up clicking links and before you know it you have lost an hour. I lost more time than I should looking around what is one of my all time favourite sites and its virtual museum.

I also found this link on which may be of interest to anyone doing research in to rural networks

I am now focused on just reviewing each site and not getting drawn in to the links. Once I have checked each site listed on the archive page I can go back and revisit the sites I have added and look for other local sites that are linked from them to add to the list.

I’ve not yet found a hyperlocal site for Dunfermline, home of the funders of Local Web List, Carnegie UK Trust, if anyone can help with this or any other Scottish hyperlocal sites that are not yet on the map, feel free to jump in and add them for us, you shouldn’t be able to add any duplicates.


Updated Search

We’ve updated the search on the site so it includes meta data. WordPress doesn’t search this data, tags & categories, out of the box so we’ve added it in.

So if you want to search for sites built on a certain platform type the platform in to the search box and hit enter, the same goes for networks.

We have removed the links to Networks & Platforms from the side bar as a result of this update. If you have any problems with the search then please let us know.

How to approach white spaces in the #hyperlocal map – Wakefield experiment

Carnegie UK Trust is supporting Talk About Local in improving the map of UK hyperlocal sites.  Despite having hundreds of entries, the map when we rehosted it had some curious white spaces – it can’t be right that Devon only had five sites listed nor that towns such as Wakefield had none, nor Scotland about eight.  So Mike and I at Talk About Local have been trying several methods of filling in blanks to help the hyperlocal community link up and to help those interested in finding a local site.

We’ve had some success which we are writing up and I’ll kick off with a desk based exercise/experiment to see what we could find for Wakefield, prompted by a Leeds hyperlocal lunch.

I have no hyperlocal contacts in Wakefield so started with some random Googling for Wakefield blogs, Wakefield civic news etc – ideally I am looking for sites that link to others locally with a civic feel.  Then ricocheted around web sites or reports about the local civic society, rotary, local history, genealogy, lists of Yorkshire lifestyle bloggers, a Wakefield hashtag and other groups until  I start to turn up web media with a hyperlocal feel.

In this case Unique Wakefield – a lovely service three years old now that promotes local independent traders.  It’s not a hyperlocal news-type site but is distinctive and genuinely local.  Their website is fine but as with most modern web operations the Facebook page and Twitter feeds contain real gems.

I start to google around things that pop up on Unique Wakefield and follow some links from Twitter etc. This then gives me other things to Google and I stumble across a news report about Horbury Village where someone is clearly trying to promote what goes on there.  This leads to a Horbury Village website which is very much a traditional parish site – from there though I get to the Facebook group and Twitter feeds and also an unexpected Twitter hashtag for Horbury.   The social media are rich.  Again this is a common pattern in 2015 – a trad website very much for reference that shows up in Google but new updates and action on Facebook and Twitter.

Looking at the map to work out where Horbury is I see a large parks complex to the North East and Google for Thornes Park hoping to find a ‘Friends of’ website, which turns up but as a bonus covers several local parks.   But I find another entry in google for Thornes Park on a much older platform which was nominated for an award for e-government in 2006.  Back tracking to the parent site takes me to Wakefield District Community Online – a platform apparently provided by the council. The full site list operating on this CMS seems to be over 100 local sites.  This will no doubt lead me off to record others to go on the map.  I have also mailed the local library to see if they have a list of links, which is common.

About 45 minutes work enables me to start to fill in a blank spot on the map and I still need to put a couple of hours more into Wakefield alone.  It’s very labour intensive and is hard to distil out as a method.  I was pleased that when I talked this through in a hyperlocal lunch in Leeds, no one there had come across the sites I had mapped.  Emma Bearman observed that this could be automated somehow – maybe running a gazetteer of place names against Twitter , looking for repetitions and hashtags to give a list for manual inspection.  I agree but am not sure how to go about it yet.  I shall write up shortly my experience with Devon and Somerset where I was able to add dozens of local parish sites.

Site ‘Groups’

We found a field in the Openly Local data that attributed some sites to networks such as LocalPeople etc.  We have as an experiment revealed this for a while as a category selector at the bottom of the front page.  However most of these sites are now redundant and so we shall remove this when we cleanse the data.