Meta
Probing exit poll probabilities
This post is a couple of days late, since the elections were on Saturday, but well, I felt I should get to grips with the theory before making any bold statements. So for the past two days I’ve been solidly on Wikipedia learning about standard error and margin of error, testing values and looking up probabilities on the normal distribution tables.
Just when I’d more or less grokked it, I thought it might be wise to Google the “poll margin of error” to make sure I was definitely doing it right. Google Suggest offers the option of adding “calculator” to the end of my query, so I do. Then I realise I’ve just spent hours learning the theory behind this neat and easy to use calculator. Fail 
So, onto the post, shall we?
The question I’m going to answer may have crossed your mind recently:
“How likely is it for the exit poll to predict the wrong candidate?”
Here’s an appropriate snippet from the exit poll release:
Random sample of votes from first 24 hours of voting
The 24 hour thing complicates life because the first 24 hour votes may not be representative of the other voting days.
In a previous draft of this post I went into mind numbing detail explaining how to discern and proportionally apply the difference in voting patterns before and after the 24 hour period. After all of that, I ended on a sentence that pretty much said: “I currently don’t know the change in voting patterns, so I’ll assume it to be 0 for the rest of the post”.
Right, job’s a good’un.
Next up, inputted into our statistical equations, if we were getting really serious we’d want to know the population size from which the sample was taken, that is, how many votes were cast in total in the 24 hour period for each position. However, this theoretically shouldn’t affect the value of the equations if we’re using the right methods. The statistical calculator assumes a multiplication factor of 1.0 and the one I worked out was 0.94, so that’s good enough for me.
Now we enter the part which used to be littered with calculation print-outs, equations and useful numbers, but we don’t need that anymore thanks to the calculator.
The next bit of useful info from the exit poll data is the sample size used, which is 100.
To give a sense of scale, my estimated number of presidential votes on the first day was 1773. In general, the larger the sample size in relation to the total vote count, the more accurate the poll.
Fun fact #1:
Given the sample size is fixed at 100 for all positions, and the presidential race is likely to get the most votes, it follows that the presidential exit poll is the least accurate of them all.
The sample size isn’t the whole story, because in STV some votes can get discarded along the way. In this case, we had 40 votes to Hutchings, 34 to Ngwena, and 26 discarded.
Now we have that, we can enter it into the calculator, obtaining a standard deviation of 0.59. The calculator looks this up in the normal distribution for us and returns a value of 55.5 percent. If I’ve interpreted the calculator correctly, this value applies to the range outside the two votes (outside 54% and 46%), so there is 55.5% chance that the value falls outside that range.
If that doesn’t mean anything to you then that’s fine because it doesn’t mean much to me either, so I’ll have a go at rewording it. In doing so, I will have to take the assumption we can apply the same analysis about the 46% mark as the calculator did about the 50% mark. Though this is not strictly true, they’re close enough for that to work.
Now the rewording: If Ngwena’s vote goes more than 4% in the right direction away from the predicted vote, he has a chance at winning. This tells us that 55% was our lower bound of confidence on Hutchings winning. Now, if we take the two tails of the distribution on either side of that, and only look at one of them, we get the probability that the poll was in fact incorrect and should have shown a Ngwena win.
This is 27.8%, giving another interesting stat.
Fun fact #2:
The poll was only enough to show Hutchings having a 72% chance of winning.
(The reason this is an approximation is because one of the sides of the distribution will actually be a tiny bit smaller than the other)
Lastly, we do a bit of retro-analysis:
“How likely is it for Ngwena to have won by as much as he did, just going by the exit poll?”
Ngwena actually won by 58% to 42%, which is a complete reversal from what the poll predicted, so the probability will be even smaller.
(Another assumption here is that the ratio of discounted votes remained the same in the real election as it was in our approximated poll)
58% against 42% is a 16% difference between candidates, which is double the 8% difference found in the poll, so I’m able (I think…) to simply multiply our standard deviation value by 2 (0.59 times 2 is 1.18).
This gives us fun fact #3:
There was an 11.9% probability for Ngwena to win by as much as he did from the poll prediction.
As noted earlier, this is all assuming the Monday votes are representative of those on later days, which could very well not be the case. In fact, judging by these stats I’m pretty certain the voting patterns did change over the different days. Approximately 90% certain in fact
In truth, randomly sampling a poll for which you have all the data is very much an exercise in pointlessness. Nonetheless, some info is better than none, and it keeps tensions at nerve-breaking peaks, so all in all I’m still in favour of having exit polls.
I’d really like to suggest that the union draw the sample from the entire data set though, it doesn’t really serve anyone to do it this way. In fact I can’t help but feel a niggle of doubt when confronted with a stat that says there is a 90% probability that this outcome shouldn’t have been predicted by the poll, and pulling the sample representatively will help to quell any accuracy doubts in future years.
If someone at Nouse remembers I think it’d be a swell idea for us to release an accuracy value along with the exit poll release next year too.
Well that’s it folks, hope you found this as much fun as I did, lol. If you’re a bit of a mathematician and think I did something wrong, please use the comment box!
Statisticalities
First off, tonight’s Hustings was genuinely the most enjoyable night I’ve had in some time. You’re bound to take that with a pinch of salt, being the unsolicited opinion of a great night out from an out and out nerd, but ask anyone who saw it and I think they’ll agree.
Attached to that preamble, I invite any unfortunate souls who missed it to check out the YSTV recordings which are now present on the YSTV site (of course), and our own election candidates page.
Now, for the statistically hungry, I have good news: you broke a record!
The comment count on our live blog for tonight’s Hustings is currently sitting at 195, which hands down beats previous articles.
You might have guessed this already, or possibly don’t care very much, or perhaps both, so I now hastily change tack and re-theme this post with a list of our most commented-on articles:
Top ten most commented articles on Nouse:
195 – Hustings Live – Tuesday
173 – Welfare issues at the centre of recent UGM motions
166 – Pirate Scott defies odds and sweeps to presidentia…
149 – Anti-BNP rally held in York
115 – ISA President accused of imposing ‘biased’ penalti…
101 – Anti-BNP campaign provokes battle over legality of…
95 – Controversial Gaza UGM Motion narrowly passes desp…
92 – YUSU criticised over decision to remove charity fr…
92 – Where the real power lies?
85 – Tim Ngwena wins 2009/10 YUSU Presidency – How it h…
I’ll admit to a bit of luxurious disappointment that we didn’t break the 200 barrier outright, but am heartened there are still big events ahead in the coming days and we could just make it still.
And lastly, budged along to the bottom of the post, I end on a grand finale of raw numbers for you to explode in ecstacy over.
This funny looking picture shows the number of open connections to the website. Normally, the website receives its traffic spread out all over the space-time continuum, so open connections at any time tend to be never more than 4 or so.
The liveblog changes that by keeping the connection open until you close the page, so you see above, we had roughly 40 people connected at any one time.
The interesting thing, and I’m rediscovering this here, is that on conferring with our Google Analytics stats, the average time spent on each visit to the page was no more than 5 minutes, meaning those 40 people were continually changing at quite a high rate (one every seven seconds).
One last gobsmacker of a statistic before I really must go:
Since the liveblog received 1,500 pageviews, it follows that a whopping 7,500 minutes of your time was spent on that page.
Well done to Charlotte, Laura, and Victoria for blogging, and Nousies Hannah, Hannah, and Camilla for mining quotations.
Nouse on the Mobile
Phew… production days are busy! Thanks to everyone for keeping the discussion lively here on the web, and keep it up!
At time of writing I’ve still got a few things to finish off, like adding author photos for our beautiful writers, but am pleasantly interrupted from all this by an email from Nouse Web Developer Andrew Fairbairn.
It’s done. Culminating months of hard work and overcoming of technical hurdles by Andrew, I’m incredibly excited to introduce to you Nouse’s very own mobile website!
The new site can be found at m.nouse.co.uk and will work on both computer and mobile – though obviously it’s been designed and optimised for the mobile.
We hope you like it and find it useful, but if you happen to cast a critique-al eye at it and find something amiss, we have surely obviated the need for a frown to cross your face.
In other words, please send your feedback to mobile@nouse.co.uk
Although we could automate feedback by writing programs to send ourselves mail, we’ve tried it and it’s not quite the same, so we gladly await yours instead.
As you can tell from the paragraph(s) above, I’ve gradually slipped back into nerd mode, and so I’ll take that as my cue to stop talking.
Feel free to end here with a massive grin on your face and, of course, go on the mobile site!
Passing on the baton
It’s all over. 4 years at York, and so much has changed in that 4 years, and so much will continue to change. As I was leaving the YSTV studio on Friday for the last time, after signing out my last ever key to the Nouse office, the workmen were putting up a sign in the Goodricke porters’ lodge: “Department of Mathematics. James College”. When Hes East is complete, the Uni will be unrecognisable to me if I ever come up and visit, my department won’t be next to the library any more, and Nouse might have moved into bigger offices in Langwith along with YUSU (here’s hoping). And yet, I will look back fondly on those things I remember which even people now don’t – living in Alcuin when it was a building site, the fountain between Vanbrugh and Physics, being able to quickly hop between Alcuin and Langwith without having to run down a massive hill and cross a road.
But I digress. I’m very nostalgic and sad about leaving, but I know I’ve got an awesome team to take my place. With the difficulties of holding by-elections over the summer, and with me technically no longer a student and no longer able to hold an officership in Nouse, Ali “Fantasy Football” Clark is Acting Technical Director until next term’s elections – and with the huge success of Fantasy Football and winning the technical achievement YUMA, Nouse.co.uk will surely continue to grow on to bigger and better things.
I’m not completely walking away just yet though – I have plenty of time this summer, and a few ideas I’d like to see in motion (including one suggested by a very drunk deputy politics editor in Ziggy’s back in autumn term) will come to fruition by next year.
Looking back over the year it’s staggering to see how far we’ve come: collaborating with other media societies, a redesigned website with a minisite for every major event, and according to Google Analytics an increase of 79% in visits to the site.
Now, this may be “journaljism”, blowing my own trumpet, but I’m pretty pleased with my achievements, but of course I couldn’t have done it by myself. Henry, for understanding the importance of the website and making sure it gets the right amount of emphasis, Jim, for putting up with my constant “This might be an interesting story” phone calls and e-mails, Mike Tomasello, Ali Clark and Alex Muller for turning up to and putting up with my 12 hour hackathons to get those new minisites up and running, Jenny O’Mahony for teaching my how the Nouse site worked when I took over, Tom Brearley for helping rebuild the Nouse network, and for everyone in the office who suffered when the part I tried to do without Tom’s help collapsed, Anna for being incredibly patient with all of those videos, and Matthew for those 2am discussions on the landing. And of course, Lida, Beth and Emily for editing all those pieces that people sent to go online, and to be there every Tuesday when everyone else was relaxing for producing the paper whilst we had our own copy, paste and resize image frenzy to get everything online for people to comment on. All of the writers, many of whom I never met, for actually getting the important stuff, content, and of course, all of our commenters. Some comments made me laugh (one of my favourite ones was left by someone who used the e-mail address “youknowwhothisis@northwood.com” in the comment field), some made me cry, but most were good and important discussion. I like to think that some of the issues brought up by our articles and the subsequent discussion actually led to real change.
Now that’s over, I can cover the thing I actually wanted to write about: Every term I sent out a “top 10″ to the Nouse mailing list covering what the most read articles of the term are. Mainly because I find these kind of stats interesting, but I know some of you do too.
So here’s the top 10 of this entire year:
10. JSoc accuse University staff member of anti-Semitism – I personally was surprised that this was the only article where Nouse was accused of being irresponsible to appear in the top 10, but our first article in this countdown was a highly controversial article from February following accusations made by JSoc against a staff member causing an absolute flood of comments and 2700 views.
9. Friday’s coverage of Roses – the smallest day of Roses takes quite a low position in the top 10 with only 2730 views.
8. The sad news of Ron Weir passing away – it’s no surprise such a moving story as this appears so high, with both current students, staff and alumni adding their consolations to their article.
7. Police search student accommodation – one of the biggest news stories of the year, and it happened outside of term time too, meaning Nouse.co.uk was the only site to cover it, bringing us 2830 views.
6. York’s sex trade – probably slightly surprising to see a feature from 2007 this high, but further investigation reveals the reason. Nouse has a very strong Google presence (incoming links from national papers, other blogs, etc all make it), which actually makes this article second on Google for “sex in York”. What can we say, sex sells – 2910 views this year in fact.
5. Sunday at Roses – no surprise to see the second biggest day of Roses take number 5 with 2930 views.
4. The gender-neutral toilets UGM – covering a very controversial topic and an absolutely staggering number of comments, this is also the most commented piece on the entire website, 173 comments and 3180 views.
3. The liveblog of Roses Saturday – another unsurprising one here, the biggest day of Roses takes it place at number 3 with 3950 views.
2. The sports team’s interview with John Motson – largely helped by a huge amount of incoming links from major forums and the “blogosphere”, this tallied up nearly 4780 views.
1. YUSU elections live blog – I guess this is no real surprise, it’s the biggest single night of the year for campus media, but there it is – at over 9000 views.
And that’s it. Bye.
Downtime
As any webmaster will know, downtime is evil. It’s to be avoided at all costs. When it happens, you look bad, you lose visitors, and if you’re a commercial entity, advertising revenue.
The Nouse site was down for a few hours around midday today, and although as much I hope to be able to say “it’s not my fault” and blame external factors, it was. Problems I did not foresee during what should have been a lightweight procedure essentially crippled our MySQL server, until I figured out the problem and solved it.
Last night I started an overhaul of our databases. Due to a legacy issue with WordPress, our database stored UTF-8 text, but it was identified as Latin-1. This caused a huge number of internal problems – backups are double encoded UTF-8 (which is horrible, every quote is replaced with three characters) and the databases actually held incorrect data, which made doing anything outside of WordPress complex – MySQL would convert any UTF-8 data incoming to Latin-1, despite us actually wanting UTF-8.
The solution was deceptively simple, and thanks to blogger Alex King, I solved this last night.
The motivation to solve this is that this character encoding issue was blocking me from introducing something I wanted to do, and mentioned a few entries ago – migrating the original Nouse site over to WordPress.
It’s this migration that took the site down today. Our central “wp_posts” table hits 213 MB in size, with the vast majority of that being fulltext search indexes for the search function and the related posts plugin. It’s these indexes that started to kill the site. An INSERT was taking up to 2 minutes, which was fine on the wp_posts table in our testing site because nothing else was trying to read that at the same time, but what I did not realise was when I ran the operation live on the server, it would start blocking normal wp_post reads, tying up Apache processes and quickly hitting the connection limit.
It took me quite a long time to realise the cause of the long inserts. Once I did, a simple ALTER TABLE wp_posts DISABLE KEYS solved it, re-enabling at the end to enable the fulltext searching once again. This change was dramatic, instead of running for an hour and only getting a sixth of the way through the conversion, by the time I had switched back to the other terminal, the job had completed.
So, I apologise for the downtime earlier – I had a few people e-mail me to complain about the site being down, but think of the silver lining on the cloud – we now have an additional 800 or so articles in our archive, stretching back to 2003.
The articles back then are written in a different style, less “broadsheet”, more of an “us against them” mentality (Nouse wasn’t redesigned into it’s current form until 2005 – look at the old PDFs for comparison!), but for people like me who like seeing the way things were and with posterity, they’re fascinating.
Lies, damn lies and statistics
Okay, so it’s been quite some time since my last blog, but I’ve got a very good excuse – I’m a finalist, and essay deadlines and looming exams have got in the way of doing Nouse things.
Nothing too exciting has happened technically with the website since Roses, with the exception of the EU Elections tab, which was quite a simple job with HTML and then abstracting some of the lightbox code from the YUSU Elections minisite to make it reusable.
One thing I’ve talked about in the past with non-Nouse people is how we monitor things, both the Nouse server, and the website itself.
The website itself has nothing particularly fancy, Google Analytics is the workhorse and what we use as our official statistics, but we also have AWStats running which analyses log files hourly and makes the result available, instead of the lag that GA has.
One tool I’ve seen that does look particularly interesting is Woopra (Tim Ngwena tweeted about this a while back), which offers live tracking and analysis. This would be particularly useful for watching the statistics on live events such as Roses or Elections, but I’m unsure if putting yet more JavaScript in the site is such a good idea, and Google Analytics is far too good a tool to let go, so at the moment I’m not implementing it.
So far, so normal, however as Nouse operates from Linode as opposed to a typical shared server, there is a lot more that we can monitor, and it is normally these tools that interest other people.
I’ve been using MRTG now for quite some time on my own servers and graphs are quite nifty.
MRTG is designed to map the traffic of a router, but can easily be extended by a script that provides two numbers (representing “in” and “out” bits per second, but the labels can easily be changed), a hostname and an uptime. There’s a number of fairly normal things I track, such as CPU usage, RAM usage, HDD usage, etc, but there are a few things in particular that are more important for Nouse:
The first is the number of Apache requests being handled. This is crucially important, especially considering how heavyweight WordPress is in memory use (just under 60 MB per request) and keeping this below the point at which we start swapping and performance degrades past a useful point is very important.
The second thing I track is MySQL activity – both read (SELECT) and write (UPDATE, INSERT, DELETE, etc), and is the best indicator of server load. Looking at the year graph also shows us a few interesting events:
The blue line represents SELECT statements, and the green line UPDATE, INSERT, etc. The interesting events you can see on this graphs is when I configured WP Super Cache which made a massive difference to server load, and also when I deployed the new site, which contained an annoying bug resulting in far too many INSERT statements (the Popular This Week plugin used to count a view as when a post is displayed, but WordPress thinks a post is displayed whenever it’s headline is shown, resulting in an INSERT for every headline, rather than one per pageload). Spotting once this bug was fixed is also quite easy on that graph. Another interesting point is the increased amount of queries since May, which I hope means that the momentum we gathered during Roses has continued through the rest of the term.
The final thing I track was developed for the live blog and shows how many people are currently connected. Here’s the graph from Friday:
The little blip the day before is a result of testing, and at first it might seem that the numbers here are quite low, especially as Google Analytics recorded over 2000 views for the live blog on this day alone, but I suspect that means that user behaviour is that for something like Roses, most people just popped in to check, and then moved on, coming back later, rather than leaving it to run all the time.
Of course, statistics are useless without anything to compare it against. The Yorker no longer publish their web stats on their advertising page (we do, and I try to keep them up-to-date, especially considering the massive growth we’ve seen since the new site launched in February), but if they’ve not grown since they last published them (which seems unlikely, of course), our statistics compare favourably.
(Also, in an unrelated note, I will be filming some of the Apprentice York stuff for YSTV tonight, and one of the remaining teams include Nouse Comment Editor Charlotte Hogarth-Jones, and News Correspondent Holly Hyde, so I feel obliged to recommend that you all pop down to Derwent tonight to support Mr York 2009, where Nouse editor Henry James Foy will be competing against a few people, including regular Nouse commenter, Dan Taylor).
The Nouse custom plugins
Just a short blog post this time.
Before, I’ve promised to release some of the custom plugins we use here at Nouse, and I’ve now gone and got the plugins listed on WordPress’s Extend directory for people to get.
We have 7 custom plugins and 13 standard plugins, and now 5 of those plugins will be available for you to use on your own blog if you want to. The other 2 custom plugins are very unlikely to see the light of the day, but are probably uninteresting (not that all the ones that are being released are interesting). They’re a ‘Media Charter Compliance’ plugin that sends YUSU an e-mail whenever a new news, sport or comment article is posted, and the second is our YSTV integration plugin.
However, the other 5 are available now from WordPress.org:
Our plugin to have images attached to articles and dynamically resize them on the backend.
A plugin that exports RSS feeds in a JSON form.
The Popular This Week widget.
A plugin that lets you have images associated with tags.
Our live blogging plugin. I made a screencast to demonstrate how to use this too.
So, short and sweet, but this is the giving back to the open-source community I talked about back in my first blog post.
Reflecting on Roses
It’s been a while since my last post on this blog, but as you can imagine, it’s all been go for team Nouse tech.
The week leading up to Roses was a big hard job for me and the rest of the development team, but one we accomplished well, matching the amazing work put in by the sports team, the photo team, Anna Bucks and the YSTV team, and everyone else who helped out, who as well as running the coverage and live blog were putting together a newspaper at the same time.
Everything went very smoothly technically, which was my main concern, especially with launching such a big new feature as the streaming live blog. I’m fairly sure I scared everyone in the Nouse office on Thursday night as I was shouting “fuck you” at my laptop repeatedly, normally whilst debugging in IE6 (approximately 15% of our readers still read in IE6, presumably due to computing services not upgrading the machines on campus), but it all seemed to work pretty well at the end of the day.
The drive behind launching the streaming live blog was two fold, firstly to reduce the load on Apache over the weekend and secondly to get the news out faster than anyone else. During testing, the news would slide in on the connected clients before the “Post successful” screen even loaded on our back end.
If you missed it, we also launched an image slideshow, more dynamic than the simple “picture of the day” we used for Elections, that hooked into the wonderful Flickr API to get the photos the photographers uploaded immediately. Mike Tomasello was responsible for that one.
So, back to live blogging, the crown in the jewel as it were. Now I’m going to get a bit technical, but there’s diagrams so hopefully not too technical (a rough version of this was in the office whiteboard, and Henry seemed to understand it with copious amounts of hand waving).
The bit of technology that made it all possible was Meteor Server. Basically, it serves a web page that appears to take ages to load – but web browsers will deal with web pages as they come in, not wait for it to load, so when you connect to the live blog, this web page opens in a hidden part of the page and then whenever Meteor receives a new message from us, it then sends it to all the connected browsers as a little bit more of the page. And it’s even more clever, because if this doesn’t work in your browser, or behind your connection, it “polls”, which means instead of having something open constantly, every 5 seconds it reconnects and asks if there’s any more data to receive. So at worst you’ll have a 5 second delay.
This is the software Twitterfall use to power their stuff, so big thanks to them for pointing me in the right direction and giving me some helpful tidbits.
Setting up Meteor was a bit challenging, least of all because it’s documentation isn’t great, but there also were bugs with it – chiefly the security settings it tried to set are invalid for .co.uk domains (like Nouse), but the bug didn’t prove to be too hard to track down and fix, and being good open source consumers, we gave the patch back to the community.
So, I wrote a WordPress plugin to link this all together (as well as pushing to Twitter), which works roughly as the diagram shows:
Once the message (actually an encoded JSON string) is received by the browser, the Meteor client library passes it to our event handler, written by Ali ‘Fantasy Football’ Clark, which does the magic animation stuff.
Then, of course, Richard Mitchell suggested we should have comment streaming. Another late night later, liveblogging.php catches comments as they’re posted (if they’re from an authenticated user, or when they get approved by a moderator otherwise) and then streams them out using Meteor (but on a different channel) and the event handler posts them in the appropriate place.
It’s not perfect, there’s still a couple of bugs to work out (the ‘Your comment is awaiting moderation’ message will stay on your screen even after moderation – but that’s not too difficult to solve), and sending a comment still requires a POST request – I’d prefer to have it AJAX’d, but the core of it is there.
So that’s it! That’s our secret behind live blogging. The plugin will be released as open source as soon as I’ve abstracted some Nouse specific stuff away.
Roses was a fantastic experience, it’s a shame I couldn’t get out and see any of it (I was the one responsible for updating the official score on roses2009.com, whose API fed the Nouse site, as well as being fed in part by the Nouse site itself), but as score central was YSTV, I did get to see whatever they caught on camera. I also got to see the reactions of LU Tube (Lancaster’s TV station) who were sharing the studio as our points just went up, and up, and up.
A Window to the Past
Tom Scott once said that campus has a three-year memory. And he’s right. There’s no-one at Nouse who’s been at York longer than 4 years and as this generation graduates in 10 weeks, gone will be the last few Nouse-ites who don’t pay top-up fees, those who remember every bar being open every night of the week.
Of course, it’s our job to report it here and now, but what about for future generations? Our Deputy Editor wrote a piece for the elections supplement about the history of YUSU, but there’s wasn’t really much to go on in the way of history, except for old copies of Nouse (which I’ve been informed by the library are available 24/7 in the microfilm room!) and the folklore page, which I’m informed isn’t the most accurate and unbiased account in the world.
Marie Thouaille of The Yorker recently wrote a comment piece on the Claudia Lawrence case, and she seems to have fallen into the same trap.
People disappear in films all the time, but in real life – or rather, in our comfortable uni-bubble, the reality of which we may question – we don’t expect such brutal realities to hit
It does happen, and it does happen with a worrying regularity, whether it’s unfortunate deaths, or something more serious. The OU murder of the 90s is just as shocking, but after that campus returned to normality as Marie suggested, and I’m sure it will do again. But, we should learn from the past to avoid it repeating itself.
Foy once called me a bastion of posterity (after I shouted at him for uploading an election photo of the day in a way that deleted old ones), and that’s something I’ve tried to implement into the Nouse site. The ‘Related Posts’ plugin tries to show similar stories, or stories in the same form from the past, and the ‘From The Archives’ section on the News page, but they only go back as far as early 2005, there’s still 41 years of Nouse history buried away…
The original Nouse site was launched in 2003, but for whatever reason the content on it was never moved when the site moved to WordPress in 2005. I’ve recently found a backup of the original site, and eventually will write a converter to move it to the current WordPress site, but reading back some of it is particularly interesting, especially in light of Summer Ball. Our front page from June 2003 discussed moving the Gradball (as it was called then) from Elvington Airfield to the Racecourse, and some resulting controversy.
And not just the content is interesting, PDF archives are too showing the design of the paper over the years, the images and look of the time. One of the biggest criticisms I received internally about the new site was the lack of a PDF archive, so it’s been brought back and up-to-date, but without the tedious uploading process by predecessors had – opening the PDF that went to the printers into Photoshop, resizing and thumbnailing, and then manually uploading all the pages and images.
Apparently a couple of hours of work, all replaced by a Python script. (Linked here, for the curious).
So our archives from here on forward will be on the web for all to see, but there’s still some work to do with the past. Talking to the Borthwick Archives proved enlightening, but they brought up a good point which may prove the archives to be impossible – copyright. It’s unlikely the original authors of the time would have consented their work to go on the web – especially as the web didn’t exist for a lot of the time back then.
Of course some people are glad that campus only has a 3 year memory. That their deeds will go forgotten and they can perhaps leave a youth with mistakes behind them. But the Internet – Facebook, Flickr, Twitter – is changing all that and making one big collective memory. One I want to contribute to. Of course, that means my colleagues who actually do the journalism have a greater responsibility than ever.
Looking inside the magic box
Running a website the size of Nouse.co.uk presents a number of challenges, least of all in infrastructure – providing not only the main website and the features that support it, but staging areas, development areas, SVN version control, development tools, issue tracking, wikis…
Added features bring unexpected consequences, and even unexpected uses, as the York students behind Twitterfall, Nouse Computing Officer Tom Brearley and David Somers, discovered following yesterday’s problems on the Telegraph website. The problems we experience at Nouse aren’t on the same scale, but they, and the solutions to them, are interesting. Tweets like JTownend’s show that people like to know what’s going on behind the scenes, and Brearley’s reply of “magic” leaves a lot to be desired.
I’ve an advocate of free and open-source software, where people can see the work of others (the “source code” – the recipes that make up the software) and modify and improve them as they see fit, and hopefully allowing others to do the same, resulting in a great group-think resulting in an excellent overall product.
Blogs are intended to be highly personal. They’re not “print quality”, they don’t go through an editor first, and they don’t even get sub-edited (too much). They’re aimed to be a direct tap to the heart of the blogger, and collectively the blogs coming on Nouse over the following weeks will cover a much wider subject matter than articles have covered before. And by restricting one into a particular blog, you just need to pick and choose which ones you want.
In this blog I am to cover the interesting problems that come up with running Nouse.co.uk, and demystify some of the behind-the-scenes “magic” – putting this knowledge and my solutions into the so-called “blogosphere” for other people to benefit from. Eventually (when the code is relevant and mature enough) I also aim to release some of the custom code that’s gone into Nouse.co.uk as WordPress plugins. Approximately half of the plugins installed into our WordPress were written internally, with the rest being ones from WordPress.org.
I should probably finish off by saying who I am, as my name normally doesn’t grace the side of an article. As it says up there, I’m the Technical Director of Nouse. I’m in charge of leading a team of 3 amazing developers, pretty much everything to do with the website short of the actual words on the page, and I moderate most of the comments that come through. I’m also pretty outspoken, have an opinion on everything and try to get involved with campus politics – voting, attending hustings, former JCRC officer and even had a couple of UGM motions passed. If you know me, it’s probably because my name appears in the comments section under too many of the articles on the site.
This blog is aimed to be a bit technical, so I hope that’s not geeked anyone out too much!