Twitter Analytics Using TCAT and Tableau, via Gawk and BigQuery

I’ve previously introduced my TCAT-Process package of helper scripts (written in Gawk), which take exports of Twitter data from the Twitter Capture and Analysis Toolkit (TCAT), developed by the Digital Methods Initiative at the University of Amsterdam, and convert them to a format that is best suited to using the data in the analytics software …

Anyone for Some Quick Crowdsourced Twitter Research?

Taking a quick break from the AoIR 2015 liveblogging at snurb.info: today’s presentation by Fabio Giglietto, Luca Rossi and Jiyoung Kim got me thinking. They built on a paper by Stefan Stieglitz and me which compared some basic properties of a large number of hashtag datasets (and some keyword-based datasets, too), and used these to …

Metrics for Analysing Twitter Communities, Using TCAT and Tableau

This post builds on the new approach to transforming Twitter datasets generated by the TCAT tracking tool for analysis in Tableau which I’ve introduced in my recent posts. Often, we will be interested in exploring the structure of Twitter communities as they form around given hashtags or keywords – for instance to examine whether they …

Using Gawk to Prepare TCAT Data for Tableau, Part 2

In my previous post, I introduced a new set of Gawk scripts to extract a range of additional information from standard TCAT datafiles, in order to enable their use for data exploration, analysis, and visualisation in Tableau. After running the TCAT-Process scripts, we now have the following datafiles: datafile.csv – the original TCAT dataset (a …

Using Gawk to Prepare TCAT Data for Tableau, Part 1

Much of the research we’ve presented on this site over the years has built on yourTwapperkeeper, our trusty old tool for gathering Twitter data. But yTK isn’t the most modern of platforms any more, provides only a very limited user interface, and gathers only a fraction of the full metadata payload which the Twitter API …

Rebooting ATNIX, using MySQL and Tableau

During 2012 and 2013 I published a more-or-less-weekly overview of the news sharing patterns in the Australian Twittersphere, the Australian Twitter News Index (ATNIX), which I also crossposted to my column at The Conversation. Technical issues and the call of other commitments have forced me to put ATNIX on the backburner for some time, but …

First Steps in Exploring the Australian Twittersphere

Twitter is widely used in Australia, but we don’t actually know such a great deal about the structure and dynamics of the Australian Twittersphere. Back in 2011/12, our research began to identify Australian Twitter users and map their follower/followee connections in order to develop a better understanding of the structure of the network and from …

Layers of Communication on Twitter

If you’ve been wondering about the lack of news on this site – yes, we’re still here, and if we haven’t posted since the start of the year it’s because behind the scenes we’ve been busy preparing for a number of major new research projects that are about to start, as well as crunching some …

Introducing Twitter Follower Accession Graphs

It’s been uncommonly quiet on this site for the last few weeks, as we’ve been busy with various projects, and overseas for a range of research activities. This may well prove to be the calm before the storm, though – we have a number of very exciting new social media research outcomes in the pipeline, …

Moving Politics Online: How Australian Mainstream Media Portray Social Media as Political Tools

(by Theresa Sauter and Axel Bruns) Difficult as it may be to believe, we’re still almost three months out from the likely date of the next Australian federal election; campaigning during this time will become even more frenzied than it has been to date. A sea of speculation, controversy, and crisis surrounds the polls, and …