Using Gawk to Prepare TCAT Data for Tableau, Part 2

In my previous post, I introduced a new set of Gawk scripts to extract a range of additional information from standard TCAT datafiles, in order to enable their use for data exploration, analysis, and visualisation in Tableau. After running the TCAT-Process scripts, we now have the following datafiles: datafile.csv – the original TCAT dataset (a …

Using Gawk to Prepare TCAT Data for Tableau, Part 1

Much of the research we’ve presented on this site over the years has built on yourTwapperkeeper, our trusty old tool for gathering Twitter data. But yTK isn’t the most modern of platforms any more, provides only a very limited user interface, and gathers only a fraction of the full metadata payload which the Twitter API …

Rebooting ATNIX, using MySQL and Tableau

During 2012 and 2013 I published a more-or-less-weekly overview of the news sharing patterns in the Australian Twittersphere, the Australian Twitter News Index (ATNIX), which I also crossposted to my column at The Conversation. Technical issues and the call of other commitments have forced me to put ATNIX on the backburner for some time, but …

First Steps in Exploring the Australian Twittersphere

Twitter is widely used in Australia, but we don’t actually know such a great deal about the structure and dynamics of the Australian Twittersphere. Back in 2011/12, our research began to identify Australian Twitter users and map their follower/followee connections in order to develop a better understanding of the structure of the network and from …

Introducing Twitter Follower Accession Graphs

It’s been uncommonly quiet on this site for the last few weeks, as we’ve been busy with various projects, and overseas for a range of research activities. This may well prove to be the calm before the storm, though – we have a number of very exciting new social media research outcomes in the pipeline, …

A First Look at Twitter Ratios: Rise of the Lurkers?

For this blog, I have used data sets which include the first million and recent million IDs discussed in recent posts, in addition to new data from our CCI Data Scientist Troy Sadkowsky covering ID’s between 1000000000-1,000,999,999 (1 million Ids) and 1,001,000,000-1,011,000,000 (10 million Ids). This data covers both the first few months of Twitter …

New M/C Journal Article on the Harmful Effects of Twitter’s Corporate Policies

October has been a busy month – as those of you who follow my personal research blog may already have seen, our Mapping Online Publics colleague Tim Highfield and I have presented several papers at the Association of Internet Researchers (AoIR) conference in Manchester and the European Communication Conference (ECREA) in Istanbul. Additionally, I also …

A Quick Recap of Twitter Research Approaches

Any self-respecting Internet researcher will already be aware that the Association of Internet Researchers (AoIR) is the place to keep track of what’s cutting edge in the field. On the AoIR mailing-list, there’s been an interesting discussion over the last few days about the available tools for tracking, capturing, and analysing Twitter data – and …

Tracking Twitter: yourTwapperkeeper and Other Options

Well, it was about time we refreshed the look of Mapping Online Publics – hope you like it… A few posts on recent activities have been held up while we were working on the redesign, so expect some catch-up articles over the next little while. First off the rank: with one of our colleagues at …

Twapperkeeper and Beyond: A Reminder

Those of you who have followed our adventures in Twitter research for some time now will know that we’ve relied to a significant extent on Joe John O’Brien III’s excellent Twapperkeeper as a tool for capturing tweets. Twapperkeeper (as a stand-alone, free Web-based service) no longer exists in its original form, however – though some …