[Air-L] archival/search tools for Sina and Weibo?

[Liao, Wang]

Hi Prof. Aaron,

I’ve some experiences in scrawling Sina Weibo, but I think it might be hard to download the data you mentioned currently.

I’ve conducted a research about Sina Weibo last year, when I wrote a scrawler script to parse the page and download it through the search function of Weibo. However, months later, Sina changed the feature of its search function, which only retrieves 50 recent pages. So it is impossible now to retrieve all data about that accident, since they were buried in later irrelevant posts. Meanwhile, using the API of Weibo is also impossible to get that data, because the search API is only open to Sina’s partners, while average individual developer can not use it. 

Therefore, I am afraid that there is no third-party tools can get the data you mentioned (since it must use the search API). But since you know all relevant users, you might write some script to download all the posts from those users through parsing the page, and to filter out relevant posts.

Regards,
Liao, Wang


On Oct 2, 2011, at 12:12 PM, Aaron S. Veenstra wrote:

> Hi all,
> 
> I have a student who is interested in studying how Chinese microblogs
> compared to traditional media outlets in covering the July 23 train
> accident in Wenzhou. She's been looking at Sina and Weibo, and having
> trouble getting any kind of helpful search results -- the best she's
> been able to do is trace back individual users who were know to have
> posted about it from the present back to late July. I've used Twapper
> Keeper to get a targeted archive of old tweets, and I'm wondering if
> anybody knows of any similar third-party tools for Sina and/or Weibo.
> Anything that would allow for a keyword/date search would be perfect.
> 
> TIA,
> 
> Aaron
> 
> -- 
> Aaron S. Veenstra
> Assistant Professor, Southern Illinois University Carbondale
> School of Journalism || 1234 Comm Building
> asveenstra at siu.edu || manytoomany.com
> _______________________________________________
> The Air-L at listserv.aoir.org mailing list
> is provided by the Association of Internet Researchers http://aoir.org
> Subscribe, change options or unsubscribe at: http://listserv.aoir.org/listinfo.cgi/air-l-aoir.org
> 
> Join the Association of Internet Researchers:
> http://www.aoir.org/