Twitter Analytics Using TCAT and Tableau, via Gawk and BigQuery

I’ve previously introduced my TCAT-Process package of helper scripts (written in Gawk), which take exports of Twitter data from the Twitter Capture and Analysis Toolkit (TCAT), developed by the Digital Methods Initiative at the University of Amsterdam, and convert them to a format that is best suited to using the data in the analytics software …

Metrics for Analysing Twitter Communities, Using TCAT and Tableau

This post builds on the new approach to transforming Twitter datasets generated by the TCAT tracking tool for analysis in Tableau which I’ve introduced in my recent posts. Often, we will be interested in exploring the structure of Twitter communities as they form around given hashtags or keywords – for instance to examine whether they …

Using Gawk to Prepare TCAT Data for Tableau, Part 2

In my previous post, I introduced a new set of Gawk scripts to extract a range of additional information from standard TCAT datafiles, in order to enable their use for data exploration, analysis, and visualisation in Tableau. After running the TCAT-Process scripts, we now have the following datafiles: datafile.csv – the original TCAT dataset (a …

Using Gawk to Prepare TCAT Data for Tableau, Part 1

Much of the research we’ve presented on this site over the years has built on yourTwapperkeeper, our trusty old tool for gathering Twitter data. But yTK isn’t the most modern of platforms any more, provides only a very limited user interface, and gathers only a fraction of the full metadata payload which the Twitter API …

Rebooting ATNIX, using MySQL and Tableau

During 2012 and 2013 I published a more-or-less-weekly overview of the news sharing patterns in the Australian Twittersphere, the Australian Twitter News Index (ATNIX), which I also crossposted to my column at The Conversation. Technical issues and the call of other commitments have forced me to put ATNIX on the backburner for some time, but …