{"id":1022,"date":"2012-01-09T10:20:35","date_gmt":"2012-01-09T00:20:35","guid":{"rendered":"http:\/\/www.mappingonlinepublics.net\/dev\/2012\/01\/09\/twapperkeeper-and-beyond-a-reminder\/"},"modified":"2012-04-10T14:16:34","modified_gmt":"2012-04-10T04:16:34","slug":"twapperkeeper-and-beyond-a-reminder","status":"publish","type":"post","link":"https:\/\/mappingonlinepublics.net\/dev\/2012\/01\/09\/twapperkeeper-and-beyond-a-reminder\/","title":{"rendered":"Twapperkeeper and Beyond: A Reminder"},"content":{"rendered":"<p>Those of you who have followed our adventures in <em>Twitter<\/em> research for some time now will know that we&#8217;ve relied to a significant extent on <strike>Joe<\/strike> John O&#8217;Brien III&#8217;s excellent <em><a href=\"http:\/\/twapperkeeper.com\/\">Twapperkeeper<\/a><\/em> as a tool for capturing tweets. <em>Twapperkeeper<\/em> (as a stand-alone, free Web-based service) no longer exists in its original form, however &#8211; though some of its functionality for creating <em>Twitter<\/em> archives appears to have been subsumed into the for-pay services available as premium offerings from <em><a href=\"http:\/\/hootsuite.com\/\">Hootsuite<\/a><\/em> &#8211; and so we&#8217;ve been getting the occasional <a href=\"http:\/\/www.mappingonlinepublics.net\/dev\/2010\/10\/20\/dynamic-networks-in-gephi-from-twapperkeeper-to-gexf\/#comment-18422\">inquiry<\/a> about what to do now.<\/p>\n<p><!--more--><\/p>\n<p>Some months ago, I published a quick post to outline how we&#8217;ve transitioned from <em>Twapperkeeper<\/em>(<em>.com<\/em>) to the open-source solution <em><a href=\"https:\/\/github.com\/jobrieniii\/yourTwapperKeeper\">yourTwapperkeeper<\/a><\/em>, which offers comparable functionality as a Web package which users are able to install on their owns servers, and the start of a new year seems like a good point to reiterate this, as well as to add a few further pointers. So:<\/p>\n<ul>\n<li><em>yourTwapperkeeper<\/em> does pretty much exactly what <em>Twapperkeeper<\/em> did, and provides data in almost the same format. For the purposes of using the Gawk scripts which much of our work is built on, though, we need CSV or TSV files in the original <em>Twapperkeeper<\/em> format, and I&#8217;ve made available a small modification for <em>yourTwapperkeeper<\/em> which generates them. <a href=\"http:\/\/www.mappingonlinepublics.net\/dev\/2011\/06\/21\/switching-from-twapperkeeper-to-yourtwapperkeeper\/\">More details here.<\/a> <\/li>\n<li>Your Web server must be running 24\/7 if you want to capture comprehensive datasets from <em>Twitter<\/em>. If there&#8217;s any chance that it may go down at some point (e.g. due to regular scheduled maintenance), you need to make sure that <em>yTK<\/em> is restarted as soon as the server is back up again. <a href=\"http:\/\/groups.google.com\/group\/yourtwapperkeeper\/browse_thread\/thread\/2064f9393be7c57c\">Some information on how to do so is here.<\/a> <\/li>\n<li><strong>Most importantly:<\/strong> recent changes at <em>Twitter<\/em> now require API requests to use the https (rather than plain http) protocol. This means that <font color=\"#ff0000\">(if you have an existing install of <em>yTK<\/em>)<\/font>you need to make some minor changes to the <em>yourTwapperkeeper<\/em> code <font color=\"#ff0000\">(or use <a href=\"https:\/\/github.com\/jobrieniii\/yourTwapperKeeper\">the latest version from Github<\/a>, which has the changes already built in)<\/font>; without these changes, you may only receive data from the search API, but not the (more important) streaming API, or even none at all. <a href=\"http:\/\/groups.google.com\/group\/yourtwapperkeeper\/browse_thread\/thread\/599d51e7d250a52e?pli=1\">Details on how to make these changes are here.<\/a> <\/li>\n<\/ul>\n<p>&#160; <br \/>Hope this helps. Happy <em>Twapperkeeping<\/em>!<\/p>\n<!-- AddThis Advanced Settings generic via filter on the_content --><!-- AddThis Share Buttons generic via filter on the_content -->","protected":false},"excerpt":{"rendered":"<p>Those of you who have followed our adventures in Twitter research for some time now will know that we&#8217;ve relied to a significant extent on Joe John O&#8217;Brien III&#8217;s excellent Twapperkeeper as a tool for capturing tweets. Twapperkeeper (as a stand-alone, free Web-based service) no longer exists in its original form, however &#8211; though some &hellip; <\/p>\n<p class=\"link-more\"><a href=\"https:\/\/mappingonlinepublics.net\/dev\/2012\/01\/09\/twapperkeeper-and-beyond-a-reminder\/\" class=\"more-link\">Continue reading<span class=\"screen-reader-text\"> &#8220;Twapperkeeper and Beyond: A Reminder&#8221;<\/span><\/a><\/p>\n<p><!-- AddThis Advanced Settings generic via filter on get_the_excerpt --><!-- AddThis Share Buttons generic via filter on get_the_excerpt --><\/p>\n","protected":false},"author":2,"featured_media":1324,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[175,113,8],"tags":[143,9,298,111],"class_list":["post-1022","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-capture","category-tools-2","category-twitter","tag-archiving","tag-twapperkeeper","tag-twitter","tag-yourtwapperkeeper","entry"],"_links":{"self":[{"href":"https:\/\/mappingonlinepublics.net\/dev\/wp-json\/wp\/v2\/posts\/1022","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/mappingonlinepublics.net\/dev\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/mappingonlinepublics.net\/dev\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/mappingonlinepublics.net\/dev\/wp-json\/wp\/v2\/users\/2"}],"replies":[{"embeddable":true,"href":"https:\/\/mappingonlinepublics.net\/dev\/wp-json\/wp\/v2\/comments?post=1022"}],"version-history":[{"count":0,"href":"https:\/\/mappingonlinepublics.net\/dev\/wp-json\/wp\/v2\/posts\/1022\/revisions"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/mappingonlinepublics.net\/dev\/wp-json\/wp\/v2\/media\/1324"}],"wp:attachment":[{"href":"https:\/\/mappingonlinepublics.net\/dev\/wp-json\/wp\/v2\/media?parent=1022"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/mappingonlinepublics.net\/dev\/wp-json\/wp\/v2\/categories?post=1022"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/mappingonlinepublics.net\/dev\/wp-json\/wp\/v2\/tags?post=1022"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}