originally by lingpipe
Here’s a 2 page write-up of one of the models I’ve been looking at for evaluating data annotation in order to evaluate coding standards and annotator sensitivity and specificity:
* Carpenter, Bob. 2008. Hierarchical Bayesian Models of Categorical Data Analysis.
I’ve submitted it as a poster to the New York Academy of Sciences 3rd Annual Machine Learning Symposium, which will be October 10, 2008.
Please let me know what you think (carp@alias-i.com). I didn’t have room to squeeze in the more complex model that accounts for “easy” items. This model and the one for easy items derive from the epidemiology literature (cited in the paper), where they’re trying to estimate disease prevalence from a heterogeneous set of tests. I’ve added some more general Bayesian reasoning, and suggested applications for annotation (though Bruce and Wiebe were mostly there in their 1999 paper, which I cite), and for training using probabilistic supervision (don’t think anyone’s done this yet).
I’m happy to share the R scripts and BUGS models I used to generate the data, fit the models, and display the results. I’d also love to know how to get rid of those useless vertical axes in the posterior histogram plots.
Hierarchical Bayesian Models of Categorical Data Annotation
Posted by jeffy
Posted on 8:20 PM
with No comments
loading..
Popular Posts
-
Resources about lucene Resources Introductions The API documentation contains a short and simple code example that show...
-
Just read a post from http://blog.bigml.com/2013/02/21/everything-you-wanted-to-know-about-machine-learning-but-were-too-afraid-to-ask-pa...
-
汉字编码问题 下面是搜集的多篇关于汉字编码问题文章的合集,相信你的问题一定包含在其中,如果没有请留言,一起把这方面的内容补充全。 一、汉字编码的种类 汉字编码中现在主要用到的有三类,包括GBK,GB2312和Big5。 1、 GB2312又称国标码 ,由国家标准总...
-
Repost from http://terrytao.wordpress.com/advice-on-writing-papers/ There are three rules for writing the novel. Unfortunately, no on...
-
Sent to you by jeffye via Google Reader: (title unknown) via 异度空间——Sue's Cabinet by 苏绥 on 3/10/09 Books on Infor...
-
This plugin can be used to read a RSS feed and transform it into a custom piece of HTML. Setup <!DOCTYPE html> <html> <...
-
This is a very useful list of surveys from Doug Oard , which would be definitely helpful for those who want to enter this territory. I...
-
SIGIR 2014 accepted Full Papers Included here is a tentative list of the full papers and their allocation into sessions. Note: titles, a...
-
This a nice post from http://jeroenjanssens.com/2013/09/19/seven-command-line-tools-for-data-science.html . If you play with data, these ...
-
After installing Scipy, when I import optimize from Scipy, the following error occurs. Traceback (most recent call last): File &q...
0 Comments:
Post a Comment