Meta
The Nouse custom plugins
Just a short blog post this time.
Before, I’ve promised to release some of the custom plugins we use here at Nouse, and I’ve now gone and got the plugins listed on WordPress’s Extend directory for people to get.
We have 7 custom plugins and 13 standard plugins, and now 5 of those plugins will be available for you to use on your own blog if you want to. The other 2 custom plugins are very unlikely to see the light of the day, but are probably uninteresting (not that all the ones that are being released are interesting). They’re a ‘Media Charter Compliance’ plugin that sends YUSU an e-mail whenever a new news, sport or comment article is posted, and the second is our YSTV integration plugin.
However, the other 5 are available now from WordPress.org:
Our plugin to have images attached to articles and dynamically resize them on the backend.
A plugin that exports RSS feeds in a JSON form.
The Popular This Week widget.
A plugin that lets you have images associated with tags.
Our live blogging plugin. I made a screencast to demonstrate how to use this too.
So, short and sweet, but this is the giving back to the open-source community I talked about back in my first blog post.
Reflecting on Roses
It’s been a while since my last post on this blog, but as you can imagine, it’s all been go for team Nouse tech.
The week leading up to Roses was a big hard job for me and the rest of the development team, but one we accomplished well, matching the amazing work put in by the sports team, the photo team, Anna Bucks and the YSTV team, and everyone else who helped out, who as well as running the coverage and live blog were putting together a newspaper at the same time.
Everything went very smoothly technically, which was my main concern, especially with launching such a big new feature as the streaming live blog. I’m fairly sure I scared everyone in the Nouse office on Thursday night as I was shouting “fuck you” at my laptop repeatedly, normally whilst debugging in IE6 (approximately 15% of our readers still read in IE6, presumably due to computing services not upgrading the machines on campus), but it all seemed to work pretty well at the end of the day.
The drive behind launching the streaming live blog was two fold, firstly to reduce the load on Apache over the weekend and secondly to get the news out faster than anyone else. During testing, the news would slide in on the connected clients before the “Post successful” screen even loaded on our back end.
If you missed it, we also launched an image slideshow, more dynamic than the simple “picture of the day” we used for Elections, that hooked into the wonderful Flickr API to get the photos the photographers uploaded immediately. Mike Tomasello was responsible for that one.
So, back to live blogging, the crown in the jewel as it were. Now I’m going to get a bit technical, but there’s diagrams so hopefully not too technical (a rough version of this was in the office whiteboard, and Henry seemed to understand it with copious amounts of hand waving).
The bit of technology that made it all possible was Meteor Server. Basically, it serves a web page that appears to take ages to load – but web browsers will deal with web pages as they come in, not wait for it to load, so when you connect to the live blog, this web page opens in a hidden part of the page and then whenever Meteor receives a new message from us, it then sends it to all the connected browsers as a little bit more of the page. And it’s even more clever, because if this doesn’t work in your browser, or behind your connection, it “polls”, which means instead of having something open constantly, every 5 seconds it reconnects and asks if there’s any more data to receive. So at worst you’ll have a 5 second delay.
This is the software Twitterfall use to power their stuff, so big thanks to them for pointing me in the right direction and giving me some helpful tidbits.
Setting up Meteor was a bit challenging, least of all because it’s documentation isn’t great, but there also were bugs with it – chiefly the security settings it tried to set are invalid for .co.uk domains (like Nouse), but the bug didn’t prove to be too hard to track down and fix, and being good open source consumers, we gave the patch back to the community.
So, I wrote a WordPress plugin to link this all together (as well as pushing to Twitter), which works roughly as the diagram shows:
Once the message (actually an encoded JSON string) is received by the browser, the Meteor client library passes it to our event handler, written by Ali ‘Fantasy Football’ Clark, which does the magic animation stuff.
Then, of course, Richard Mitchell suggested we should have comment streaming. Another late night later, liveblogging.php catches comments as they’re posted (if they’re from an authenticated user, or when they get approved by a moderator otherwise) and then streams them out using Meteor (but on a different channel) and the event handler posts them in the appropriate place.
It’s not perfect, there’s still a couple of bugs to work out (the ‘Your comment is awaiting moderation’ message will stay on your screen even after moderation – but that’s not too difficult to solve), and sending a comment still requires a POST request – I’d prefer to have it AJAX’d, but the core of it is there.
So that’s it! That’s our secret behind live blogging. The plugin will be released as open source as soon as I’ve abstracted some Nouse specific stuff away.
Roses was a fantastic experience, it’s a shame I couldn’t get out and see any of it (I was the one responsible for updating the official score on roses2009.com, whose API fed the Nouse site, as well as being fed in part by the Nouse site itself), but as score central was YSTV, I did get to see whatever they caught on camera. I also got to see the reactions of LU Tube (Lancaster’s TV station) who were sharing the studio as our points just went up, and up, and up.
A Window to the Past
Tom Scott once said that campus has a three-year memory. And he’s right. There’s no-one at Nouse who’s been at York longer than 4 years and as this generation graduates in 10 weeks, gone will be the last few Nouse-ites who don’t pay top-up fees, those who remember every bar being open every night of the week.
Of course, it’s our job to report it here and now, but what about for future generations? Our Deputy Editor wrote a piece for the elections supplement about the history of YUSU, but there’s wasn’t really much to go on in the way of history, except for old copies of Nouse (which I’ve been informed by the library are available 24/7 in the microfilm room!) and the folklore page, which I’m informed isn’t the most accurate and unbiased account in the world.
Marie Thouaille of The Yorker recently wrote a comment piece on the Claudia Lawrence case, and she seems to have fallen into the same trap.
People disappear in films all the time, but in real life – or rather, in our comfortable uni-bubble, the reality of which we may question – we don’t expect such brutal realities to hit
It does happen, and it does happen with a worrying regularity, whether it’s unfortunate deaths, or something more serious. The OU murder of the 90s is just as shocking, but after that campus returned to normality as Marie suggested, and I’m sure it will do again. But, we should learn from the past to avoid it repeating itself.
Foy once called me a bastion of posterity (after I shouted at him for uploading an election photo of the day in a way that deleted old ones), and that’s something I’ve tried to implement into the Nouse site. The ‘Related Posts’ plugin tries to show similar stories, or stories in the same form from the past, and the ‘From The Archives’ section on the News page, but they only go back as far as early 2005, there’s still 41 years of Nouse history buried away…
The original Nouse site was launched in 2003, but for whatever reason the content on it was never moved when the site moved to WordPress in 2005. I’ve recently found a backup of the original site, and eventually will write a converter to move it to the current WordPress site, but reading back some of it is particularly interesting, especially in light of Summer Ball. Our front page from June 2003 discussed moving the Gradball (as it was called then) from Elvington Airfield to the Racecourse, and some resulting controversy.
And not just the content is interesting, PDF archives are too showing the design of the paper over the years, the images and look of the time. One of the biggest criticisms I received internally about the new site was the lack of a PDF archive, so it’s been brought back and up-to-date, but without the tedious uploading process by predecessors had – opening the PDF that went to the printers into Photoshop, resizing and thumbnailing, and then manually uploading all the pages and images.
Apparently a couple of hours of work, all replaced by a Python script. (Linked here, for the curious).
So our archives from here on forward will be on the web for all to see, but there’s still some work to do with the past. Talking to the Borthwick Archives proved enlightening, but they brought up a good point which may prove the archives to be impossible – copyright. It’s unlikely the original authors of the time would have consented their work to go on the web – especially as the web didn’t exist for a lot of the time back then.
Of course some people are glad that campus only has a 3 year memory. That their deeds will go forgotten and they can perhaps leave a youth with mistakes behind them. But the Internet – Facebook, Flickr, Twitter – is changing all that and making one big collective memory. One I want to contribute to. Of course, that means my colleagues who actually do the journalism have a greater responsibility than ever.
Looking inside the magic box
Running a website the size of Nouse.co.uk presents a number of challenges, least of all in infrastructure – providing not only the main website and the features that support it, but staging areas, development areas, SVN version control, development tools, issue tracking, wikis…
Added features bring unexpected consequences, and even unexpected uses, as the York students behind Twitterfall, Nouse Computing Officer Tom Brearley and David Somers, discovered following yesterday’s problems on the Telegraph website. The problems we experience at Nouse aren’t on the same scale, but they, and the solutions to them, are interesting. Tweets like JTownend’s show that people like to know what’s going on behind the scenes, and Brearley’s reply of “magic” leaves a lot to be desired.
I’ve an advocate of free and open-source software, where people can see the work of others (the “source code” – the recipes that make up the software) and modify and improve them as they see fit, and hopefully allowing others to do the same, resulting in a great group-think resulting in an excellent overall product.
Blogs are intended to be highly personal. They’re not “print quality”, they don’t go through an editor first, and they don’t even get sub-edited (too much). They’re aimed to be a direct tap to the heart of the blogger, and collectively the blogs coming on Nouse over the following weeks will cover a much wider subject matter than articles have covered before. And by restricting one into a particular blog, you just need to pick and choose which ones you want.
In this blog I am to cover the interesting problems that come up with running Nouse.co.uk, and demystify some of the behind-the-scenes “magic” – putting this knowledge and my solutions into the so-called “blogosphere” for other people to benefit from. Eventually (when the code is relevant and mature enough) I also aim to release some of the custom code that’s gone into Nouse.co.uk as WordPress plugins. Approximately half of the plugins installed into our WordPress were written internally, with the rest being ones from WordPress.org.
I should probably finish off by saying who I am, as my name normally doesn’t grace the side of an article. As it says up there, I’m the Technical Director of Nouse. I’m in charge of leading a team of 3 amazing developers, pretty much everything to do with the website short of the actual words on the page, and I moderate most of the comments that come through. I’m also pretty outspoken, have an opinion on everything and try to get involved with campus politics – voting, attending hustings, former JCRC officer and even had a couple of UGM motions passed. If you know me, it’s probably because my name appears in the comments section under too many of the articles on the site.
This blog is aimed to be a bit technical, so I hope that’s not geeked anyone out too much!