originally by lingpipe
Here’s a 2 page write-up of one of the models I’ve been looking at for evaluating data annotation in order to evaluate coding standards and annotator sensitivity and specificity:
* Carpenter, Bob. 2008. Hierarchical Bayesian Models of Categorical Data Analysis.
I’ve submitted it as a poster to the New York Academy of Sciences 3rd Annual Machine Learning Symposium, which will be October 10, 2008.
Please let me know what you think (carp@alias-i.com). I didn’t have room to squeeze in the more complex model that accounts for “easy” items. This model and the one for easy items derive from the epidemiology literature (cited in the paper), where they’re trying to estimate disease prevalence from a heterogeneous set of tests. I’ve added some more general Bayesian reasoning, and suggested applications for annotation (though Bruce and Wiebe were mostly there in their 1999 paper, which I cite), and for training using probabilistic supervision (don’t think anyone’s done this yet).
I’m happy to share the R scripts and BUGS models I used to generate the data, fit the models, and display the results. I’d also love to know how to get rid of those useless vertical axes in the posterior histogram plots.
Hierarchical Bayesian Models of Categorical Data Annotation
Posted by jeffy
Posted on 8:20 PM
with No comments
loading..
Popular Posts
-
Resources about lucene Resources Introductions The API documentation contains a short and simple code example that show...
-
We examine top Python Machine learning open source projects on Github, both in terms of contributors and commits, and identify most popula...
-
汉字编码问题 下面是搜集的多篇关于汉字编码问题文章的合集,相信你的问题一定包含在其中,如果没有请留言,一起把这方面的内容补充全。 一、汉字编码的种类 汉字编码中现在主要用到的有三类,包括GBK,GB2312和Big5。 1、 GB2312又称国标码 ,由国家标准总...
-
RT: http://lingpipe-blog.com/2011/03/11/what-is-a-mathematical-model/ What is a (Mathematical) Model? By Bob Carpenter I was sh...
-
cython function in ipython %load_ext cythonmagic In [4]: %%cython ...: def fib(int n): ...: cdef int a,b,i ...: ...
-
Type in a terminal window: gs -sDEVICE=bbox -dNOPAUSE -dBATCH file.pdf (or file.ps) you must have ghostscript installed of course. This c...
-
Logistic Regression by Any Other Name LingPipe Blog I (Bob) have been working on logistic regression. In particular, multinomial logistic r...
-
After installing Scipy, when I import optimize from Scipy, the following error occurs. Traceback (most recent call last): File &q...
-
This is a very useful list of surveys from Doug Oard , which would be definitely helpful for those who want to enter this territory. I...
-
Just read a post from http://blog.bigml.com/2013/02/21/everything-you-wanted-to-know-about-machine-learning-but-were-too-afraid-to-ask-pa...
0 Comments:
Post a Comment